Cohere, a startup developing large language models to rival those of OpenAI, today announced the launch of a non-profit research lab: Cohere For AI. Cohere, led by Google alumnus Sara Hooker, says Cohere Labs will work to solve some of the industry’s toughest challenges by contributing “fundamental research” to the open source community.
†We are very excited to lead a new non-profit AI research lab as we continue to broaden how and where research is done. There is so much to discover and our focus will be on open collaboration and contributing to fundamental research,” Hooker told Vidak For Congress via email. “At the same time, an important part of our work will be to expand the community and help educate the next generation of talent, by creating new entry points to work on core research. †
There has long been a concern within the AI community that not enough money is being set aside for AI research outside rich companies. One study found that ties to companies — whether funding or membership — in AI research grew significantly from 2008 to 2019. Another found that Google parent company Alphabet, as well as Amazon and Microsoft, spent as many as 52 tenures between 2004. employed track AI professors. and 2018, removing these aspiring teachers from academic and nonprofit work.
Concentrating power within corporations has some obvious drawbacks, but one of the most alarming is that it tends to underemphasize values such as benevolence, fairness and inclusion on the research side. Speaking to Wired for a 2020 piece, a number of experts point out that corporate AI projects have led to an “unscientific fixation” on projects only possible for those with access to high-performance data centers. Regardless of the domain, work within companies is often closely guarded and takes years to see the light of day – if ever it does.
†Our agenda is focused on advancing advances in machine learning questions alongside community-based research,” Hooker said.We also want to have a proactive research agenda so we can identify big challenges before they become problems that we need to fix retroactively. We are focused on a variety of different disciplines to work, for example, on reducing bias, and a very important part of the research is AI security and the robust use of models.”
Another core component that Cohere For AI hopes to build is access to computer resources, Hooker said, and in particular it helps researchers better use “cutting-edge” models to help develop their work.” as evidenced by the trends in language models (i.e. AI systems that understand and generate text). Just a few years ago, creating a highly advanced language model required huge computational resources. But now, thanks to academic breakthroughs and the work of the open source community , the barriers to entry are much lower than they used to be.
Road to non-profit
Backed by AI celebrities, including Pieter Abbeel, co-director of UC Berkeley’s AI lab, Cohere was founded in 2019 by a family tree team that included Aidan Gomez, Ivan Zhang and Nick Frosst. Gomez co-authored the academic paper “Attention Is All You Need,” which introduced the world to a fundamental AI model architecture, the Transformer. (Among other high-profile systems, OpenAI’s GPT-3 and Codex are based on the Transformer architecture.) Zhang, along with Gomez, is a contributor to For.ai, an open AI research collective that includes data scientists and engineers.
“For.ai is designed to help novice enthusiasts better connect with more experienced researchers,” Hooker said. “Many of the founding members went on to do PhDs or work in academic or industrial labs. At the time, For.ai was one of the first community-driven research groups to support independent researchers around the world. Now the Cohere team and its supporters are excited to reintroduce the original concept, but with more resources built from Cohere. †
According to Hooker, Cohere For AI will provide ways for data scientists to “meet and collaborate” through mentorship research opportunities, involvement in traditional conferences, and contributions to research journals. Part of this will be through promoting stewardship of open source scientific practices and “responsible” code release, as well as: support efforts that promote ‘scientific communication’ through various media, such as blog posts.
“WWe really want to shape Cohere For AI as an ambitious research lab that contributes to the research community, but also prioritizes how to better engage a diverse group of voices. We want to help change where, how and by whom research is done,” said Hooker.
Despite its lofty goals, Cohere For AI — which Cohere itself will fund — is likely to spark skepticism from researchers wary of Cohere’s corporate ties. Cohere has raised $170 million to date from institutional venture capital firms, including Tiger Global Management and Index Ventures, and has a number of ties to Google. Google Cloud AI chief scientist Fei-Fei Li and Google colleague Geoffrey Hinton were early supporters of Cohere, and Gomez and Frosst previously worked at Google Brain, one of Google’s AI research divisions. Cohere also partners with Google to train large language models on the company’s dedicated hardware infrastructure.
Google dissolved an infamous AI advisory board in 2019, just a week after its creation. And in 2020, the company fired leading AI researcher Timnit Gebru in what it believes was retaliation for sending an email to colleagues critical of Google’s management practices. Google then fired another ethicist, Margaret Mitchell, who had publicly denounced the company’s handling of the situation, and a third, Satrajit Chatterjee, after he co-authored a paper describing Google’s work by AI powered chip design systems questioned.
Paved with good intentions
In general, nonprofit initiatives to fund AI research are a mixed bag.
One of the success stories is the Allen Institute for AI (AI2), founded by the late Microsoft co-founder Paul Allen, which aims to achieve scientific breakthroughs by building AI systems with reasoning power. While not strictly non-profit, Anthropic, launched by former OpenAI executives, has raised more than half a billion dollars for research into “reliable, interpretable, and controllable” AI systems.
But for every AI2 and Anthropic, there is an OpenAI, which started out as a non-profit organization before moving to capped profits and accepting a $1 billion investment from Microsoft. Meanwhile, former Google chairman Eric Schmidt’s recently announced $125 million fund for AI research sparked new controversy after Politico reported that Schmidt wields unusually high power over the White House Office of Science and Technology Policy. (One of the first recipients, Berkeley professor Rediet Abebe, asked for her name to be taken out of consideration.)
However, some newer collectives have shown promise, most notably Gebru’s Distributed AI Research, a global non-profit AI research organization. Projects such as Hugging Face’s BigScience and EleutherAI are other strong examples of what can be accomplished with AI beyond the limits of corporate influence.
“Ultimately, it’s up to us to prove that Cohere For AI is not going to become a business over time,” Hooker said. “While Cohere For AI will rely on Cohere for resources and funding, a clear separation has been made between the two to maintain its independence as a research lab. This separation is crucial in order to continue to contribute to and serve the wider community as an independent entity. Cohere For AI is structured as a non-profit organization and is intentionally designed to openly collaborate with many different organizations. Her work will be open source to allow greater access to the wider community.”