The Spider Jockey is a terrifying but thankfully rare collusion of Skeleton and Spider to create a (mostly) superior menace.
Spiders have a 1% chance to spawn with a Skeleton riding them- and this combination makes a Spider Jockey. Spider Jockeys are a bit strange, but this is because of what exactly they are. As the Skeleton is riding the Spider, the Spider controls the movement- which means that in a well-lit area, the Spider Jockey will not chase you. However, that does not in any way stop the Skeleton from shooting arrows at you, and can sometimes make the fight that much more difficult- instead of a skeleton charging towards you and juking to the side every so often, you have to deal with a Skeleton ambling around in random directions and climbing unrelated walls.
Unfortunately, this doesn’t happen very often- instead, the Spider Jockey (which spots the player just as a Spider would- through walls and obstacles) will follow the player as soon as they get within sixteen blocks, waiting for a chance to pop out and commence a double assault. The Spider’s leaping ability is hindered by the Skeleton, but that in no way prevents it from attacking just as the Skeleton does- and at the same time, to boot. This means not only are you taking double damage, you have to kill -both- the Skeleton and the Spider to put an end to the fight. Distressingly, the health of both monsters is tracked separately, so you will need to concentrate your attack on either the Skeleton or the Spider- and your chances of being able to do that while the Jockey is in melee combat with you and bouncing around like a hazardous super ball are virtually nil.
Fortunately, there -is- a small chance the Skeleton will shoot the Spider, giving you a chance to attack the Skeleton without the Spider hindering your fight, but it’s a dicey prospect.
Wolves aren’t terribly useful either- the Skeleton has the range advantage, and is more likely to hit the Wolves first than the Spider is- and the Wolves can’t reach it while it’s sitting on the Spider, resulting most often in dead Wolves and soon a dead you. Sometimes you get lucky and the Wolves fight the Spider.... but do you really want to risk your precious Wolves for that? Didn’t think so.
It is, however, possible to Jockey-proof an area; Spider Jockeys are no more agile than regular Spiders, so overhangs and narrow corridors and holes are your friends. Curiously, if the Spider Jockey can be inspired to repeatedly try climbing into an overhang, the Skeleton can wind up suffocating as the Spider jams its head into the solid surface.
Generally, taking advantage of this weakness, assault from below (causing the Skeleton to shoot the Spider to death) or archery (specifically hitting -only- the Skeleton) are the only good ways to kill off a Spider Jockey, and trying any of these without armor to protect you is ill-advised at best.
Remember that the Spider itself is a 2-wide critter, so if all else fails, dig a hole to hide down and reach up to cut the spider open from below. Just beware of the Skeleton falling into the pit with you- that can be a terrible, horrible adventure you don’t return from.
Besides, you can’t get anything from a Spider Jockey that you can’t get from a regular Spider and a regular Skeleton with -far- less risk.