The reason is that Nethack and Angband were not designed for a commercial success.Here is why. Most (if not all) commercial titles are designed with "positive emotional feedback" in mind. That means they limit amount of frustration you get while playing the game. When you play Baldur's Gate or Diablo, the worst thing that can happen to you - you die (then you reload and try again). If you survive you can restore your powers back to normal in no time. Graphics is flashy, controls are simple. However, that takes a toll on role-playing experience you get out of the game. Nethack is less forgiving. First of all if you die - you die (you cannot reload and continue). Second, there are tons of keyboard commands you need to learn to play the game. Third, a lot of nasty things can happen to you in the game: your weapons break, armor rusts and rots, items get cursed, you lose your intrinsics, you get robbed, you starve, you choke on you food, you get electrocuted, freezed, burnt, paralyzed, blinded, polymorphed, stoned (in both senses), etc. If you end up naked deep in the dungeon - it is almost certain death. It takes guts to overcome the frustration, pull yourself together and play on.
Nethack provides more hardcore role-playing experience than commercial RPG will ever offer. It comes with the price though - you need to be hardcore role-playing gamer to enjoy it.

Same goes for Angband, ADOM, Slash'em, etc.