Recently, I have been becoming more interested in retroclones, and the OSR mentality more broadly. I can't say that it really provides a compelling alternative to modern games, Dungeons and Dragons 4th edition included (since I think that many of the changes that have occurred over the years as gaming has evolved have been good ones), but retroclones do provide what I believe is a valuable insight as to where fantasy gaming has gone over the years, and more importantly, what modern players of "old-school" games continue to value about those previous incarnations. Thanks to the "open" nature of the OSR overall, very many of these retroclones are available, free of charge, for download off of their websites. Usually there is a paid version as well if you'd prefer a "dead tree" version, which also includes art that has been removed from the free versions. Of the ones I've read, I would have to say that my favorites are Labyrinth Lord and Swords and Wizardry, which are both styled after the 0e/Basic "strain" of Dungeons and Dragons.
What I prefer about Labyrinth Lord are the multiple saves against different kinds of effects; this makes sense to me. What I don't prefer about it is that all weapons deal 1d6 damage, and that it has descending AC. I feel like we've moved beyond THAC0, as a gaming culture, and one less table to look at is always a good thing. Swords and Wizardry gives the GM the choice: all stats are marked for ascending and descending AC. S&W also has more supplemental material, at least that I've been able to find, that makes it easier to customize a game. The single save against effects seems overly simplistic to me, compared to Labyrinth Lord, but it would certainly speed things up at the table. I can't imagine it would be too difficult to translate any material for LL over to S&W, or vice versa. I still can't decide, were I to actually run a game, which of the systems I would utilize. But from a purely academic standpoint, S&W seems a slight bit easier to develop from.
What I find most compelling about the "older" editions of D&D is the ambiguity of encounters. Just because it is possible to encounter a certain kind of monster, does not at all mean that you'll actually be able to kill it, or even harm it (in the case of creatures that can only be damaged by +1 weapons, silver, etc). You might find yourself totally out of your league and have to resort to quick thinking (or even quicker feet). This is a sharp contrast to 4e, where it is a pretty safe assumption that if you encounter a creature, or group of creatures at all, chances are the encounter will be "balanced" and there will be a very high likelihood of success. Additionally, given that Hit Points are rolled up for each creature based on their Hit Dice, you may end up cleaving one creature down with one swipe, but then hammer away at another one for several turns and it doesn't appear to be worse for the wear. Further, it's highly possible that that ultra-tough kobold might not even be a kobold, but some sort of shape-shifting extradimensional predator that has been slowly feeding upon the kobolds over the past few weeks and disguising itself as one of their own to avoid suspicion. And it would take only a couple minutes to completely write up and stat out such a creature, should the PCs decide later to hunt it down and try to kill it.
Additionally, many bonuses are not awarded for inherent abilities or selected feats or class powers, but simply good gaming. You might get a flat roll against an orc if you were to just walk up to it and hit it with a club, or you might get a bonus to your hit if you spend your previous turn climbing to the top of a statue, and then jumping down on top of it with your club swinging. This is not something that is impossible in 4e, but it is something which the system seems to inherently discourage. My feelings while playing 4e have always been such that a round without taking a damage-causing action is a wasted round. Plus, errata over the past four years seems to have been in a neverending cold war with itself, ramping up both monster strength and character damage, creating what is to me a total mess, if you insist on playing by the DDI rules instead of out of the books. I do like how experience is awarded in a much more free format; the big XP bonuses are for completing an adventure or achieving a major milestone, not stomping knee-deep through the dead and becoming a one-person full service slaughterhouse. I've frequently found the 4e solution of "kill everything first and look for clues later" to be troubling, and not exactly entirely encouraging of rock-solid roleplay.
Lastly, I find that the genericity of retroclones leads, at least to me, to it being a lot easier to create a custom world. For this last reason, I will be publishing, mostly as an exercise in design, my own game world, intended to take place within the Swords and Wizardry milieu. Anything with statistics I will endeavor to present in the S&W style, with both ascending and descending AC, to preserve the modularity that that method provides.
I suppose all of this is probably a direct response to the fervor (and ire) surrounding the announcement of the as-yet-officially-unnamed 5th edition of Dungeons and Dragons, and forcing me as well to think about what I like or don't like about game mechanics and design. To me, simpler is usually better; I was never on board with AD&D, and 4th edition was a welcome alternative to the (in my opinion overly laden) 3rd and 3rd-and-a-half.