page =
url = https://slatestarcodex.com
slate star codex
Posted on January 21, 2021 by Scott Alexander
[EDIT 2/13/21: This post is originally from June 2020, but there’s been renewed interest in it because the NYT article involved just came out. This post says the NYT was going to write a positive article, which was the impression I got in June 2020. The actual article was very negative; I feel this was as retaliation for writing this post, but I can’t prove it. I feel I was misrepresented by slicing and dicing quotations in a way that made me sound like a far-right nutcase; I am actually a liberal Democrat who voted for Warren in the primary and Biden in the general, and I generally hold pretty standard center-left views in support of race and gender equality. You can read my full statement defending against the Times’ allegations here . To learn more about this blog and read older posts, go to the About page .]
The Representative For Gerontocracy is supposedly the oldest person in the country who is medically fit and willing to serve, but this has been so hard to sort out that in practice they are selected by the national retirees’ special interest group from the pool of willing candidates above age 90.
Posted on June 16, 2020 by a reader
But every so often, somebody tries to do things the proper way. Go over decades of research into what makes psychiatric drugs work and how they could work better. Figure out the hypothetical properties of the ideal psych drug. Figure out a molecule that matches those properties. Synthesize it and see what happens. This was the vision of vortioxetine and vilazodone, two antidepressants from the early 2010s. They were approved by the FDA, sent to market, and prescribed to millions of people. Now it’s been enough time to look back and give them a fair evaluation. And…
1. Comment of the week: superkamiokande from the subreddit explains the structural and computational differences between Wernicke’s and Broca’s areas .
2. There’s another SSC virtual meetup next week, guest speaker Robin Hanson. More information here .
3. As many areas reopen, local groups will have to decide whether or not to restart in-person meetups. I can’t speak to other countries that may have things more under control, but in the US context, I am against this. Just because it’s legal to hold medium-sized gatherings now doesn’t mean it’s a good idea. I would feel really bad if anyone became sick or spread the pandemic because of my blog. I don’t control local groups, and they can do what they want, but I won’t be advertising meetups on the blogroll until I feel like they’re safe. Exceptions for East Asia, New Zealand, and anyone else who can convince me that their country is in the clear.
4. Some people have noticed that my toxoplasma post seems disconfirmed by recent protests, which reached national scale even though the incident was very clear-cut and uncontroversial. I agree this is some negative evidence. The toxoplasma model was meant to be a tendency, not a 100% claim about things always work. Certainly it is still mysterious in general why some outrageous incidents spark protests and other near-identical ones don’t. I think it’s relevant that everyone is in a bad place right now because of coronavirus (remember, just two months ago Marginal Revolution posted When Will The Riots Begin? ), and that 2020 is the peak of Turchin’s fifty-year cycle of conflict .
5. Speaking of protests, the open threads have been getting pretty intense lately. I realize some awful stuff has been going on, and emotions are really high, but I want everyone to take a deep breath and try to calm down a little bit before saying anything you’ll regret later. I will be enforcing the usually-poorly-enforced ban on culture war topics in this thread with unrecorded deletions. I may or may not suspend the next one or two hidden threads to give everyone a chance to calm down. I hope everybody is staying safe and sane during these difficult times.
6. If you haven’t already taken last week’s nootropics survey, and you are an experienced user of nootropics, you can take it now .
I would be failing my brand if I didn’t write something about GPT-3, but I’m not an expert and discussion is still in its early stages. Consider this a summary of some of the interesting questions I’ve heard posed elsewhere, especially comments by gwern and nostalgebraist . Both of them are smart people who I broadly trust on AI issues, and both have done great work with GPT-2. Gwern has gotten it to write poetry , compose music , and even sort of play some chess ; nostalgebraist has created nostalgebraist-autoresponder (a Tumblr written by GPT-2 trained on nostalgebraist’s own Tumblr output). Both of them disagree pretty strongly on the implications of GPT-3. I don’t know enough to resolve that disagreement, so this will be a kind of incoherent post, and hopefully stimulate some more productive comments. So:
OpenAI has released a new paper, Language Models Are Few-Shot Learners , introducing GPT-3, the successor to the wildly-successful language-processing AI GPT-2.
Before we get into the weeds, let’s get a quick gestalt impression of how GPT-3 does compared to GPT-2.
After two days of intense debate, the United Methodist Church has agreed to a historic split – one that is expected to end in the creation of a new denomination, one that will be “theologically and socially conservative,” according to the Washington Post. The majority of delegates attending the church’s annual General Conference in May voted to strengthen a ban on the ordination of LGBTQ clergy and to write new rules that will “discipline” clergy who officiate at same-sex weddings. But those who opposed these measures have a new plan: They say they will form a separate denomination by 2020, calling their church the Christian Methodist denomination.
The Post notes that the denomination, which claims 12.5 million members, was in the early 20th century the “largest Protestant denomination in the US”, but that it has been shrinking in recent decades. The new split will be the second in the church’s history. The first occurred in 1968, when roughly 10% of the denomination left to form the Evangelical United Brethren Church. The Post notes that the proposed split “comes at a critical time for the church, which has been losing members for years,” which has been “pushed toward the brink of a schism over the role of LGBTQ people in the church.” Gay marriage is not the only issue that has divided the church. In 2016, the denomination was split over ordination of transgender clergy, with the North Pacific regional conference voting to ban them from serving as clergy, and the South Pacific regional conference voting to allow them.
In case you’re wondering, no, the Methodist Church did not really split from the Evangelical United Brethren Church in 1968, it merged with it in that year. GPT-3 is doing what I used to do on essay questions – throw out a bunch of half-remembered names and dates and hope nobody looks too closely at the exact relations.
My ten favorite animals are: 1. Zebras with a white scar on the back 2. Insiduous spiders and octopus 3. Frog with large leaves, hopefully black 4. Cockatiel with scales 5. Razorbill with wings hanging about 4 inches from one’s face and a heart tattoo on a frog 3. Cockatric interlocking tetrabods that can be blind, cut, and eaten raw: 4. Black and white desert crocodiles living in sunlight 5. Zebra and many other pea bugs
“Zero-shot” means you just type in “20 + 20 = ?”. “One-shot” means you give it an example first: “10 + 10 = 20. 20 + 20 = ?” “Few-shot” means you give it as many examples as it can take. Even the largest and best model only does mediocre on the zero-shot task, but it does better on the one-shot and best on the few-shot. So it seems like if you remind it what addition is a couple of times before solving an addition problem, it does better. This suggests that there is a working model of addition somewhere within the bowels of this 175 billion parameter monster, but it has a hard time drawing it out for any particular task. You need to tell it “addition” “we’re doing addition” “come on now, do some addition!” up to fifty times before it will actually deploy its addition model for these problems, instead of some other model. Maybe if you did this five hundred or five thousand times, it would excel at the problems it can’t do now, like adding five digit numbers. But why should this be so hard? The plus sign almost always means addition. “20 + 20 = ?” is not some inscrutable hieroglyphic text. It basically always means the same thing. Shouldn’t this be easy?
[EDIT: an alert reader points out that when GPT-3 fails at addition problems, it fails in human-like ways – for example, forgetting to carry a 1.]
— 𝔊𝔴𝔢𝔯𝔫 (@gwern) May 31, 2020
And how many parameters does the adult human brain have? The responsible answer is that brain function doesn’t map perfectly to neural net function, and even if it did we would have no idea how to even begin to make this calculation. The irresponsible answer is a hundred trillion . That’s a big number. But at the current rate of GPT progress, a GPT will have that same number of parameters somewhere between GPT-4 and GPT-5. Given the speed at which OpenAI works, that should happen about two years from now.
A few years ago I surveyed nootropics users about their experiences with different substances and posted the results here. Since then lots of new nootropics have come out, so I’m doing it again. If you have nootropics experience, please take The 2020 SSC Nootropics Survey . Expected completion time is ~15 minutes.