This is probably not an answer, and certainly not a defense of atrocities, but I’d like to provoke you a little by putting you in the shoes of the 1930s USSR:

It hasn’t even been a decade since the end of a brutal Civil War, during which the world’s first newly formed socialist state was invaded by a dozen of Western imperialist powers, and ended with a heavy toll on human lives and the destruction of huge parts of the economy.

A new form of reactionary force - fascism, embodying the most brutal form of reactionary violence, had just crushed the revolutionary movement in Italy, and is spreading across Europe. Hitler had just been made the leader of Germany, and is heavily militarizing and preparing to execute an expansionist doctrine.

War is coming. An imperialist war that threatens to be far bloodier than the Great War itself not even two decades ago, is now painting the world’s first socialist state as its target. You pleaded with the European powers to take this fascist threat seriously, but was turned down by every single one of them, who then proceeded to form military and economic alliance with Nazi Germany. It became clear to you that the anti-communist imperialist powers are dead set on destroying the Soviet Union. It should have been obvious.

A war is coming, and your country is still 50-100 years behind the Western capitalist powers. It was barely industrialized, and there simply isn’t enough productive capacity to match the output of the Western powers. The fate of the world’s first socialist state is in peril - if the Soviet Union is crushed, then all hopes would be lost. Reactionary forces would have won. This has gone beyond the survival of the nation - this is the ultimate clash between ideologies, and the survival of socialist ideas hinges upon the survival of the Soviet Union itself.

You don’t know when the war would come. Would it be in 5 years? 10 years? There is no time to waste. There is only one way out: brutal industrialization. “We are 50-100 years behind the capitalist powers, and we must make do of that in 10 years” - Stalin in 1930. This proved to be prophetic, because in exactly 10 years, Nazi Germany would launch its invasion against the Soviet Union that would end with the perishing of 27 million lives of the Soviet people.

A brutal industrialization must take place before the worst comes for us! It must not be derailed! Anyone who threatens to derail the industrialization process also threatens the survival of the nation itself. It is a time of confusion and uncertainty. You cannot know for sure who might be the saboteurs, the naysayers, the delinquents who will drag us down and set us back. There is no time for that. Would you risk having millions of lives killed in a war because you’re not sure if a few managers are innocent? There is no time to tell, and no way to know for certain.

You have to make the choice. Now. And your choice will determine whether fascism destroys an entire nation and leads to the final victory of the reactionary forces.

What would you do?

There is certainly survivorship bias because all the socialist states that had perished have merely become a footnote, that leftists lament over: “how unfortunate”. Allende in Chile tried to be democratic with his socialism, and what he invited instead was the most brutal form of fascist violence that murdered hundreds of thousands of people, and the most brutal form of economic exploitation known as neoliberalism, the effects of which had destroyed millions of families and its devastation still lingers to this day in Latin America.

It is easy to say, “if I had been in charge, I would certainly have done it better.” But that’s with the power of hindsight. But let’s say you have just achieved a socialist revolution in your country, how would you act and prepare for the coming onslaught of the counter-revolutionaries? How far are you willing to go to defend the survival of the revolution?

In a time of confusion, uncertainty, social upheaval, radical transformation of the society, the emotions, with all the flux of information that may or may not be trustworthy, if you were to make decisions which will determine the survival or the demise of an entire nation, could you really have done better?

I'd recommend "Stalin: History and Critique of a Black Legend" by Domenico Losurdo.

In it, Losudro brings up the 'thornier' parts of Stalin's history and contextualizes those moments in history. To me, he does a convincing job of explaining all of the 'Great Man' myths we hear about Stalin, and that so much of these unexamined anecdotes just lived on as myths/legends about the man.

I believe he has a few chapters about work camps and explains and contextualizes them in a nuanced way. Really enlightening book and I think it would answer some of the questions you might have about Stalin.

I think Blackshirts and Reds was a very accessible USSR book.

Most leftist books, I've found, are very well sourced.

I will say this, I get frustrated by most any anti-Stalin slander, but I also get really depressed when i look into a figure in soviet history and see "died 1938-9" because so so often it is needless. So many comrades lives wasted and in many cases indefensibly, however what is worth considering is these are the exception not the rule. Most people who got tried didn't get executed, most purged just got reassigned or kicked from the party, and the vast majority of citizens faced none of this. That doesn't make a great deal of it any less of a mistake, but it becomes a mistake among triumphs.

There is a reason so many people who got purged and executed have wiki articles, and it is not just western propaganda, it is because people in prominent and noteworthy positions were more likely to be killed or become relevant in hindsight, whereas random worker who got his pension and retired happily and voted in his soviet is not "worth" making an entry for, and for understandable reasons.

So many dissidents or executed people who get attention are artists, creatives, and this seems to imply that the purges went so deep into harmless things, but really it shows us that a great deal of the purges are part of an almost culture war or battle within an agency, a disproportionate number of those purged are public individuals. So you naturally will see more info on them.

Another part of it is that there are for sure bad actors and people operating out of malice, plenty involved in the ethnic expulsions did so with cruelty and while there are reasons for why at the highest levels say Stalin would approve of deporting so many Chinese and Korean communists from the Far East out of concerns that Japan would claim that made it legitimate territory for Manchuria to claim (as they had elsewhere), another case like that of the Ingush is far less reasonable even compared to the incorrect logic of the prior example, and carried out even more harshly often in an openly racist manner by officials. Sometimes this was ignored or approved of by higher officials, but often local powerbases protected themselves until dismantled.

We see this with Ukraine and Stalin bringing the hammer down on officials who did discriminate and mistreat people intentionally