Hang a quick right into the chaurus pen first, opening the weird chitinous gate, and you can loot the corpse of a prisoner fed to the huge bugs, as well as the chaurus themselves and a huge array of chaurus eggs for alchemical use. Immediately past the gate to the chaurus pen is the passage back to where you climbed a ledge instead of staying in the water, as well, so keep that in mind in case you come through again later.
Stay on the right hand side of the water to gather STILL MORE MUSHROOMS (as if you hadn't had enough of those already) and the contents of one of the strange clamshell-esque falmer chests, then ascend the ramp carefully to pick off yet another weapon-and-spell falmer. Up here is even more fungus, surprise surprise, and some charred skeever hide you can gather. There's a small passage up here as well, which leads to a strange room centered around a stone bridge and an altar. The room is guarded by several archer falmer and a Falmer Nightbane, which is to say, an offensive spellcaster. Take care of the lot of them (preferably the Nightbane first) and you can toss the room for- what else?- even more fungus and chaurus eggs, particularly from the pool of water under the stone bridge. Scrounge through the pockets of the sacrificed prisoner, then head up the bridge.
Here you'll find another small room full of fungus, but there are several other treats as well- bags and barrels with food in them (and, in one case, spider eggs), a weapon and a few armor pieces sitting on a table, a chest full of treasure, and a book that grants you a skill rank in Lockpicking.
There is one other passage from here, but it leads to a hole in the wall over the water near where you entered the cave, so you're not going through that yet- you didn't finish up in the room with the chaurus pen! On the upside, now you know how to leave when you're ready.
Go back through the altar room and down the ramp from the ledge in the largest cavern, and follow the right wall to get still more fungus (because you need more). A dead man named Firir lies on a pallet in the pen on the left, check him to recieve a note addressed to someone named Eyris. The first pen on the right has mushrooms (tired of those yet?) and a locked chest- though how a lock on a chitin and bone chest not only can be picked with a standard lockpick but is also made of iron and brass is beyond me.
The second cell on the right holds only an empty pallet and some unusables- a bowl, a dead lantern, another few patches of fungus. The fourth and final cell has more fungus and a dead woman, and that's it.
Head back up the ramps and out the hole in the wall if you feel like a shortcut, and then you can go- because you're done in here.