Kind of. It's still better than most RPGs in that you can actually go around stealing stuff like a thief is supposed to, but it's not nearly as rewarding and there aren't as many interesting or rare items to steal as there are in every subsequent TES game. Stealth is a very useful skill in dungeons since most of the powerful monsters can see through invisibility anyway.
Also note that there's no NPC detection system or crime reporting like there is in the later TES games. It's all up to a roll of the dice whether the psychic guards get summoned when you rifle through things, even if you're in an empty building. However, as long as you don't get damaged while the guards are chasing you, your crime is not recognized. So the typical run for a thief is to try to steal something, run from the guards until they despawn, and repeat. If you do this in a shop then you need to wait a few in-game days for the guards to despawn. Outside they'll despawn as soon as you get a short distance away from them.
There are no fences and no "stolen" items. You can stand right in front of a shopkeeper, steal a Daedric Wakizashi from a shelf, murder every single guard that spawns with arrows or spells, then sell it right back to him for around ten thousand gold. The merchant will still like you just as much as he did before and nobody will know that you've ever committed a crime. That situation is actually kind of realistic. Would you rat on a thief who slaughtered everyone that could protect you?