Not long enough reach, these mice are quick as fuck, unless you're Solid Snake or some shit, stabbing this is like stabbing a bullet.
The idea is to use a mousetrap. When the fucker is trapped there, you'll know by the short squeaks pleading for freedom as it struggles to move. What you do next is, get a large steel knife and start making small pokes around its neck. Then, as if you would be peeling an orange, dab the knife in the thing's neck close to its head, and begin moving forward and back, slightly downward with each stroke. You'll notice by this time the short squeaks have evolved into long, mad child-like screams. Keep doing the strokes, watch out for the blood that may spurt out at any given moment. The screams will fade into short, loud gurgles. Keep stroking away, moving slower and slower downward. You'll feel a cracking sound; you've snapped the thing's arteries and veins. Don't let this bother you though. About halfway cut through the neck, stroke faster and faster as you hear more gurgles and blood pools out, dripping from the knife. More crunchy noises will appear; keep stroking until you have the neck almost completely cut. You'll notice the mouse's eyes will blink, some distinct sounds of blood rushing out of the cut arteries.
Once the head is off, dispose of the carcass as you see fit. I threw it out on the street in a paper bag.
Alternatively you can go David from RE:Outbreak and tie a butcher knife to the end of a stick (like a broom) with duct tape, and try spearing the poor sap.