A Metropolitan Police spokesman said officers were patrolling the central London area about 11.30pm on Sunday when they noticed the man acting suspiciously.
When challenged by police, the man produced two knives, the spokesman added.
Armed officers responded and a Taser and police firearm were used.
London Ambulance Service attended but the man was pronounced dead at the scene at Great Scotland Yard a short time later.
No police officers or members of the public were injured during the incident.
Forensic officers at the scene of the shooting on Monday morning (PA)
The force said it was not treating it as a terrorism incident. The man has not been publicly identified.
Uniformed officers manned cordons at Scotland Place, leading to Great Scotland Yard, and on parts of Northumberland Avenue and Whitehall nearest to Trafalgar Square.
Roads coming off Trafalgar Square were still blocked this morning.
Hungerford Bridge was cordoned off overnight.
Scotland Yard sent officers to the scene at around 11.30pm
The Directorate of Professional Standards and the Independent Office for Police Conduct have been informed.
Enquiries into the incident are ongoing.
Police are continuing to investigate the incident, which is not terror-related (PA)
A Scotland Yard statement said: “Police officers were on patrol at around 23:25hrs on Sunday, 8 March when they noticed a man who they believed to be acting suspiciously.