Harry Kane fired England ahead with his tournament-leading sixth goal just before the hour in Moscow, converting a penalty after he was hauled down by Carlos Sanchez at a corner.
Gareth Southgate's team looked to be heading into the last eight in regulation time until Yerry Mina equalised with a 93rd-minute header as the giant Colombian defender scored for the third game in a row.
Neither team could find an advantage in extra-time and so a bad-tempered match went to a nailbiting penalty shootout.
David Ospina dived superbly to his left to repel Jordan Henderson's third penalty for England, but Manuel Uribe hit the crossbar before Jordan Pickford saved from Carlos Bacca.
Eric Dier drilled home the decisive spot-kick as England won for just the second time in eight penalty shootouts and reached the quarter-finals for the first time since 2006.
"To get knocked down at the end like we did with that goal in extra time, you know, it's difficult to come back from that. But we did," Dier said.
"We were ready for that. We knew what we had to do, we stayed calm, we stuck to our plan, we never panicked all the way through extra time.
"We knew that if it has to go to penalties, it has to go to penalties. We were ready for that."
James Rodriguez blow
Colombia were dealt a huge blow before kick-off as James Rodriguez failed to shake off a calf injury, the Bayern Munich midfielder not even fit enough for a place on the bench.
The 26-year-old was the Golden Boot winner with six goals in Brazil four years ago, when Colombia reached the quarter-finals for the first time.
Kane, Raheem Sterling and Dele Alli were among nine players recalled for England after Gareth Southgate rested the majority of his first-choice side for their final group game against Belgium.
Kane scored a combined five goals in the wins over Tunisia and Panama, and the Tottenham striker went close when his looping header back across goal landed on the roof of the net.
Sterling then did well to spin past the imposing Davinson Sanchez although his left-footed strike was charged down, while Kieran Trippier curled wide from a free-kick.
Wilmar Barrios was booked for leaning his head into the chest of Jordan Henderson as the game grew into an increasingly fiesty affair.
Pickford made his first save from a fizzing long-range effort from Juan Fernado Quintero, while Jesse Lingard sliced over after Colombia half-cleared a cross.
A member of the Colombia coaching staff appeared to deliberately barge Sterling as the Manchester City player headed towards the half-time at the interval, escaping with just a telling-off.
Kane made the breakthrough on 57 minutes after he was wrestled to the ground at a corner by Carlos Sanchez, the same player who was shown the first red card of the tournament in a 2-1 loss to Japan.
The England captain faced a lengthy wait as several Colombia players protested to the referee but he kept his nerve to beat north London rival Ospina, lifting his penalty just over the Arsenal goalkeeper's outstretched leg.
Alli nearly added a second when his diving header at the far post flashed over the bar, with Lingard appealing for another spot-kick after he was caught by Davinson Sanchez.
Juan Cuadrado blazed over with 10 minutes to play, and it appeared Colombia's hopes were up when Pickford made a terrific save to turn behind Manuel Uribe's spectacular long-range strike.
But Mina rose powerfully to meet the resulting corner, with Trippier only able to help the ball in via the underside of the crossbar in the third minute of added-on time.
Substitute Danny Rose almost struck an extra-time winner when his angled shot whistled beyond the far post, and Eric Dier headed over unmarked at a corner before England's luck deserted them again in all too familiar fashion.
Then, unlike so many times before, luck finally smiled on them in the shootout.