Amid the wave of resignations at OpenAI, another researcher officially announced his resignation and issued a “warning” message:
Ilya and Jan Leike's concerns are exactly my concerns, with some additional issues.
We need to do more to improve fundamental things like decision-making processes, including accountability, transparency, documentation, policy enforcement, how to use technology carefully, and measures to mitigate inequalities, rights, and environmental impacts.
The protagonist's name is Gretchen Krueger(Ge Jie for short) is an AI strategy researcher at OpenAI.
Advertisement
She joined OpenAI in 2019 and participated in the development of GPT-4 and DALL・E 2. She also led OpenAI's first company-wide “red team” test in 2020.
In addition to mentioning OpenAI’sDecision-making transparencyShe also hinted at the problem:
Generally speaking, one of the ways that tech companies disempower those who seek to hold them accountable is by creating divisions among those who raise concerns or challenge their power. I care deeply about preventing that from happening.
Including Ilya and Jan Leike, Ge Jie is the third security-related personnel to leave OpenAI this month – she submitted her resignation before Ilya and Jan Leike confirmed their resignation.
Advertisement
The third OpenAI security member to leave this month
Let’s briefly review the whole thing.
Sister Ge is an AI strategy researcher at OpenAI. Some netizens helped everyone understand the responsibilities of this position in the comment section.
Feeling that OpenAI posed a huge security risk, Ge informed the company on May 14 that she was about to leave.
After this, OpenAI chief scientist Ilya officially announced that he would leave the company to pursue “the next project.”
Following Ilya’s resignation, Jan Leike, head of super alignment, also accused OpenAI of “prioritizing products over safety.”
So, what exactly was the risk that made Sister Ge want to run away so early?
The summary is actually very simple, but also very critical: OpenAI makes decisions on safety issuesNot transparent enoughandLack of accountability for decision makers.
(AI transparency, etc.) Issues are important to everyone and they affect how the future is planned and by whom.
I want to emphasize that these concerns should not be misinterpreted as narrow, speculative, or out of touch. They are not.
In addition, Ge Jie also specifically mentioned Daniel Kokotajio, a former OpenAI employee who gave up 85% of his family assets in order to retain the right to criticize OpenAI. She expressed her gratitude for his courage, which gave her the opportunity to say what she wanted to say.
Previously, OpenAI was exposed to require signing a non-disclosure agreement when leaving, and “failure to sign will affect equity.”
The latest development of this matter was revealed by another former colleague of Sister Ge, Jacob Hilton:
After Vox's in-depth investigation was published, OpenAI contacted former employees and released them from the nondisclosure agreements they had signed.
Back to the topic of safety, Sister Ge still kept her former employer's dignity:
OpenAI is still leading this important work. I will continue to be interested and excited about it.
However, Ge Jie's departure still aroused the curiosity of more netizens:
How does OpenAI decide major safety issues when Ilya et al. are still around?
How does OpenAI currently decide safety issues when a group of important safety members have left?
Who decides OpenAI’s safety path?
According to the information released on the official website, OpenAI’s current security team is mainly divided into three parts.
Super Alignment Team: Take out 20% of the total computing power, aiming at how to control the super intelligence that does not exist yet
Safety Systems team: Focused on reducing abuse of existing models and products (e.g. ChatGPT)
Preparedness Team: Mapping emerging risks with cutting-edge models
First, the core super alignment team was launched by Ilya and Jan Leike in July 2023.
As a result, within less than a year, the two key figures left one after another, and the super team fell apart.
Secondly, Safety Systems was established in 2020. The team has four departments, including the Security Engineering Team, Risk Management Team, Monitoring and Response Team, and Policy and Compliance Team.
It was responsible for the security assessment of GPT-3 and GPT-4.
Dario Amodei, who led it at the time, later left OpenAI and founded Anthropic.
And the reason why Dario Amodei left was precisely becauseWant to build a more credible model.
Finally, OpenAI announced the establishment of the Preparedness team in October 2023.
The group, led by Aleksander Madry, director of MIT’s Center for Deployable Machine Learning, aims to “track, predict, and protect against future dangers from AI systems,” ranging from AI’s ability to “convince and deceive” humans (such as in phishing attacks) to its ability to generate malicious code.
Currently, the team focuses on model capability assessment, evaluation and internal red teaming (i.e. the attackers who conduct penetration testing on the models).
Having said so much, but——
No matter how many safety teams OpenAI has, the final decision-making power for its risk assessments remains in the hands of leadership.
Before the palace fighting scandal, OpenAI's core management team had four members: CEO Sam Altman, President Greg Brockman, Chief Scientist Ilya Sutskever, and CTO Mira Murati.
Although Ilya initially launched a “palace coup” to force Sam Altman to step down temporarily, Altman soon “returned to the palace” and it was Ilya who “disappeared” for 6 months.
After his return, Sam Altman is surrounded by basically his own people, including Jakub, who has recently replaced Ilya.
However, regarding decision-making power on safety issues, although the leadership is the decision maker, the board of directors has the right to overturn the decision.
Current OpenAI board members include Salesforce co-CEO and chairman Bret Taylor, economist Larry Summers, Quora CEO Adam D'Angelo, former Bill & Melinda Gates Foundation CEO Sue Desmond-Hellmann, former Sony Entertainment president Nicole Seligman, Instacart CEO Fidji Simo, and Altman himself.
Reference Links:
(1)https://x.com/GretchenMarina/status/1793403475260551517
(2)https://futurism.com/the-byte/openai-cryptic-warning
Advertising Statement: The external jump links contained in the article (including but not limited to hyperlinks, QR codes, passwords, etc.) are used to convey more information and save selection time. The results are for reference only. All articles in Gamingdeputy include this statement.