Then there’s the growing gulf between the haves and have-nots when it comes to the two pillars of AI, data and hardware. Data is often proprietary, such as the information Facebook collects on its users, or sensitive, as in the case of personal medical records. And tech giants carry out more and more research on enormous, expensive clusters of computers that few universities or smaller companies have the resources to access.

To take one example, training the language generator GPT-3 is estimated to have cost OpenAI $10 to $12 million—and that’s just the final model, not including the cost of developing and training its prototypes. “You could probably multiply that figure by at least one or two orders of magnitude,” says Benaich, who is founder of Air Street Capital, a VC firm that invests in AI startups. Only a tiny handful of big tech firms can afford to do that kind of work, he says: “Nobody else can just throw vast budgets at these experiments.”

Hypothetical question. Some people have access to GPT-3 and others do not. What happens when we start seeing papers in which GPT-3 is used by non-OpenAI researchers to achieve SOTA results?

Here’s the real problem, tho: is OpenAI picking research winners and losers?

— Mark Riedl : Human-Centered AI Total Landscaping (@mark_riedl) October 3, 2020