There is also the thing that the Nazi apparatus, its economy and political system wasn't going to last anyhow.

They were very inefficient, the leader principle and conflicting places of power who would by force decide who gets favors led to extreme economic inefficiency, to extreme dissatisfaction of the workforce and comparatively low innovation. One effect of this was that for investments or even unhindered continued production you had to pay party officials (often multiple) and also something more or less being Kontaktbeamte (officials and unofficial persons of political power) to not shut down your projects or letting them happen. Funfact: the Nazi police cadres were not seldom, even on local areas corrupt, which meant that you might've fared more well to seek allegiance with them by paying them off, too. This was also sometimes used as a bridge so they would harass your competition or seek reasons to destroy/imprison/expropriate them. All this also became much more visible and problematic with the "Ersatz"-Wirtschaft (substitute economy) since they couldn't get all their necessary products from the world market relatively soon, and the regular economy became riddled with inefficiencies and the need to use the third best option for production.

Besides those inefficiencies the frozen wages of workers did not lead to higher productivity and the abolishment of union laws for companies led to a disengaged workforce that was cheap, but became less and less relevant as consumers. The forced labor from prisons, camps and such was used to substitute the demand for more workers (also cause of workable men doing economic unproductive work of soldiering) - however: and this is important! The annihilation did not just happen for economic reasons, but on the contrary the extermination of groups of people who often had hundreds of years of past in German territories was driven by a will to exterminate that is a core believe of the Nazi ideology. The Shoa, the Porajmos and such are examples of planed and wilful attacks by the party and the majority society on jews, romnja and sintizze, gays, further (political) enemies like communists, socialists, anarchists, union leaders, partially liberal opposition, pacifists, church members, slavs, neuroatypical, people called "work-shy" which could be everything from unemployed to being part of the former groups or more etc. with the goal to eliminate them from Nazi-German society (often) once and for all. However especially in the cases of the first two groups there was a century old tradition of discrimination by majority society and pogroms weren't a thing invented by the Nazis, but something that continued since hundreds of years. I would like to add that there were also attacks on (female) reproduction with forced sterilizations, attacks on people in Anstalten (like asylumns, psychatric wards, orphanage hospitals, etc. etc.) going by names like operation T4 or operation Brandt which included the centrally planed and later decentrally executed extermination of life deemed by the Nazi leadership (and not seldom even by the ableist people working in those places) as not worth living.

All that I wrote to just get to the main point of the "imperial aggression of the Nazis". They managed to nearly bancrupt their state very fast. It was obvious that they would not be able to keep up even parts of their military, nor feed their population before the war. Without the war the Nazis would've been deemed the biggest economic losers of the century. They needed the loot of the other countries and expropriation of wealth and companies to keep their economy running, they still were driving their policies with having to default and being bankrupt in a very short time in the future. This aspects often gets overlooked in analysis or history about Nazi Germany, that a military machine sooner or later has to be used if it is about the biggest post of your whole economy and you are running hard debts. There was no way that economic growth would bring them out of their problems, so they had the three choices:

1. stand down and let others take over a bancrupted Germany
2. loot other countries and not pay your invoices and use military force to stay in power
3. acquire capital from the international market (only possible by changing political leadership and privatizing your economy again), which amounts to (1)