Everything I can find online suggests that they were indeed viewed as very sus before a some campaigns from nobility and wealthy farmers convinced the general populace they were okay to eat. Reasons for the susness include:

• the aforementioned nightshade connection

• potatoes aren't mentioned in the bible which means God didn't mean for us to eat them

• herbalists thought they looked kinda like leprous flesh so if you ate it you'd get leprosy