But it wasn't to be as Kane took advantage of Scotland's poor defending to tuck home with seconds remaining on the clock.
England hardly sparkled against an industrious and spirited Scotland side in the Group F qualifier but Southgate believes they showed they have the right personality to get results in testing environments.
And the former Celtic boss was clearly frustrated as he flung a bottle during an X-rated tantrum.
"To be 2-1 down in stoppage time and get that goal is special". It could be a big point for us.
"We are disappointed but we have got to be proud of ourselves".
"It is a good point away from home".
The 114th meeting of worldwide football's oldest foes was a naturally tense affair, with the hosts looking to keep their dreams of reaching Russian Federation alive as Gareth Southgate's men attempted to tighten their grip on top spot.
"To score two goals that late in the game we just have to see the game out but it was a poor pass when it was three on one in the attack and they put the ball in the box for a great finish from Harry Kane".
Fierce clashes as US-backed force battles ISIL in Syria's Raqqa
The group said SDF forces took parts of the district of Jazra, west of Raqqa, backed by USA -led air support. He urged all parties to give safe passage to those who want to leave.
"Scotland is no easy place to come, and especially when you are 2-1 down in stoppage time, it makes it that much sweeter". I've shared the leadership around and that has been important for us as a team.
And it seems his kind words prompted a positive reaction from the world of social media, with one fan writing: "Dignity, well said Wayne, back soon".
Southgate, who reserved judgement on Joe Hart's attempts to save the free-kicks, said: "We deserved to be ahead and then there was a mad minute and a half".
"I was looking to do the exact same with the second free-kick".
The former Scotland and Manchester United midfielder said he was impressed with the physical presence of Gareth Southgate's young England side - and how his men coped with it.
"The objective is to qualify and we're top of the group". We had moments where we got into really good positions in the final third and you have got to take the finish.
"It's all to play for and don't rule us out".