Intellect devourers are really scary when the party sees them coming, but that's nothing compared to when they ambush the party in the middle of the night when Slow Dave, the lovable but not very bright barbarian, is on watch and the only one awake.
If your party gets in a fight with a werebear, whether they have silvered weapons for the non-casters makes a big difference. However, even that is a bit of a moot point: while the CR formula takes party size into account, it doesn't (and can't) look at a lot of other, equally important parameters: party composition, equipment, player tactics and ingenuity, and the way you run the monsters. I believe WotC uses a much more complex method than what they put in the DMG (too complex to put in the DMG, is the issue). I assume that by 'didn't really work' you mean that the encounter was significantly harder or easier than the CR you calculated had indicated? If so, that's because CRs are, well, not very accurate. I tried calculating the CR normally in the DMG, but it didn't really work.