Make sure you have around 3000-4000 gold on hand when you enter one of the Order of No Quarter's levels. Chester, the game's merchant, is hidden somewhere in those levels and will sell one relic per level.
Just after the second checkpoint, head down the ladder to the room pictured. Leap across the platforms, timing it so the falling lava doesn't hit you. Once you're across the chasm, line Shovel Knight up with the outline on the wall and you'll be taken to a secret room. Use the upper chandeliers to get to the Blue Chest and the lower ones to get back. Chester will sell you the Flare Wand for 1000 gold.
During this level some time after the second checkpoint, you'll fall into a room with two skeletons on the ground. Shovel Pogo off one of them to gain access to the passage left. Walk on the upper path to get to the Blue Chest, where Chester will sell you the Phase Locket for 1000 gold. Use your new item to brave the bottom path and get back on track.
After the first checkpoint, there's a room with bouncing platforms on boiling caldrons and spikes. At the end of the room, bounce on one of the two enemies there to reach the platform above. Don't destroy the blocks there, instead walk over them and destroy the wall to the right. Fight the green knight in the next room and you'll find the Blue Chest and Chester, who will sell you the Alchemy Coin for 3000 gold.
This is the easiest relic to get. After the second checkpoint, you'll fight an Angler fish mid-boss with a treasure chest attached to its head. Dodge and head right in the early part of the fight. When the fight stops moving and the mid-boss opens the Blue Chest to reveal its light, that's the spot you need to attack. Once you've defeated the mid-boss, Chester will sell you the Throwing Anchor for 3000 gold.
Just prior to the third checkpoint, you'll find a destructable wall over to the left. Hit it and walk on through to the next room. Chester will sell you the Dust Knuckles for 3000 gold. You'll need to use the Knuckles in quick succession to traverse the lower path in this room for maximum treasure.
Two rooms up from the first checkpoint - in a room with a red knight - destroy the wall on the top right. Head through into the next room and hop the platforms over to the Blue Chest. Chester will sell you the Mobile Gear for 3000 gold.
After the third checkpoint (in the room with the hammer-throwing enemies), head up the ladder and destroy the wall to the left. Use the suspended snow to safely traverse the spiked platforms. Chester will sell you the War Horn for 4000 gold.
In the room above checkpoint two, there's two fans blowing air upwards that prevent you from entering the visible secret passage. Head up the ladder to the next room and fall down into the passage to gain entry. Chester will sell you the Propeller Dagger for 4000 gold.