@tszzl — page 82/103

@hargup13 well I mean it leaked a million times

its gonna be kind of fun to see who is going to just download the LLaMa weights and figure out how to RLHF them

RL on language models is surprisingly hard to get right. i think many will beat their heads against the wall and waste a lot of compute and not get very far. will be interesting

@thomas_bfr @browserdotsys just check carefully for reward hacking. it may just be improving on irrelevant dimensions

original work doesn’t require original tools https://x.com/paulg/status/1631805100384559107

@allgarbled that’s true

a simple example: 90% of the onerous regulation surrounding nuclear power is under the guise of radiation safety for power plant workers. what about when highly autonomous humanoids can patrol the plant? even regulators have a goodwill budget to play with

this argument amounts to techno pessimism imo— if innovating in a high regulatory burden environment is a constraint satisfaction problem highly intelligent AIs should be able to navigate them better https://x.com/pmarca/status/1632135984270790657

@shorttimelines @pmarca why would you do this

also the argument that healthcare and credentials are expensive due to regulation seems wrong — the demand of rich societies for healthcare and credentials is virtually infinite so ofc the prices increase over time. even in postscarcity you would expect these prices to rise!

@rcafdm https://randomcriticalanalysis.com/2020/01/31/i-created-a-primer-on-fundamental-misconceptions-about-health-care/

@RuxandraTeslo Yeah I think that’s actually likely. AI will allow new forms of onerous regulation due to better surveillance and ability to do paperwork

AIs can be creative and can make art. this was clear from the moment they beat us at Go. creativity is metaphysical but it’s also randomness mixed with success. you wouldn’t call a useless move on the Go board creative. but you can find printouts of alpha go’s famous move 37

but doesn’t the human provide the motive via a prompt? sure but michelango’s frescoes on the Sistine chapel were commissioned by the pope who I’m sure gave him quite a few prompts. The art is still divine — the prompt contains almost zero information

ok but nature and go both have simple objectives. what if art requires a complicated objective like self expression? fitting the distribution of all human data and then fine tuning on pleasing the human eye/ ear is very much not a simple function

zooming out a bit it’s also clear that dead simple processes like evolution can be creative. the human body is a work of art. the hummingbirds’ wings are a work of art. most human art is derivative of nature’s beauty which is produced by the most simple agency imaginable

moreover i would claim that everything the language model says, as a professionally trained actress, is self expression

I also see the prompt thing as an implementation detail. highly autonomous language models will say artful things without anybody asking them to. I just don’t think anyone currently has an appetite for highly autonomous language models

@LouisAnslow lol sure

@HingumTringum No

this is just incredibly low imagination it’s like saying well some states require rent seeking gas station attendants so gas cars aren’t massively impactful on the economy. having a medical doctor who is paid to press a button approving the superior decision of an ASI is fine https://x.com/ESYudkowsky/status/1438198184954580994

@jcarterwil people don’t actually have an unlimited demand for food. I’m sure the prices at the top restaurants keep going up but wanting super high end stuff is more of a personality quirk than a universal desire

@HingumTringum wrong he’s a very smart commentstor which is why it’s worth the discussion

having land in San Francisco and defending your nice views is basically a luxury that will keep increasing in price well into post-scarcity. if actual construction and transport prices are zero you can go and build a lower status city somewhere else https://x.com/oreos2002/status/1632549272401920001

@raspy_aspie metabolic privilege is obviously real

@ESYudkowsky @nptacek @davis_yoshida i do think if iterated amplification is true then understanding the basic math of PPO or data engineering or what have you buys more runway until you get to the extremely powerful misaligned model

@ESYudkowsky @nptacek @davis_yoshida if a 50% more aligned GPT-N within its capabilities envelope today leads to a 10% more aligned GPT-N+1 this is an important thing

@EvansVeris I’m praying for the San Francisco mega city to come but it’s not happening

you see it more often in the bandwagoners. they immediately see the economic value of todays models and not the real underpinnings of the project: to create beings that are intelligent, think more powerfully than Man does and live in our computers — not beasts of burden

i think lacking fundamental understanding and respect for the AGI research plan, not internalizing the idea that these are highly intelligent aliens, leads to the twin idiocies of dismissing AI risk outright and ignoring the potential moral catastrophes of training them

the models will grow from antlike to mammalian to human, and it’s not very clear where we are at any given point. we can only hope our descendants, be they human or otherwise, will forgive any moral catastrophes we may be unknowingly committing today in our near total ignorance

@ESYudkowsky speculation is still ignorance

@ESYudkowsky also, which questions have they answered? which fiction has seriously broached the questions of when and how experience, emotions, or suffering appears? I don’t see it

@SturnioloSimone @ESYudkowsky very uncharitable moral catastrophes that are happening right now can be prevented by strong AI systems and this fact is clear and near certain. the moral catastrophes of model suffering are speculative and barely broached even in fiction. it’s Pascal’s wager stuff right now

@peachblvd 😞

@ruthhook_ @growing_daniel bull signal on the shrimp suffering

@amolitor99 @SturnioloSimone @ESYudkowsky your pessimism is strange! godlike intelligences that can wipe out civilizations should also have the capacity to tremendously elevate them. whether we’re able to elicit these behaviors or not is a different question

i don’t understand the Fermi paradox solution where AI kills the host civilization and then goes totally dead silent. this seems unintuitive

in fact i think the Fermi paradox points to the idea that nobody in the galactic neighborhood ever built an unbounded optimizer. maybe there’s no such thing. it certainly makes me take ai risk less seriously but also the kardashev types beyond I less seriously

@anthrupad we are assembled and the galaxy is only 100,000 light years across

@pshc @anthrupad the copernican principle demands that we don’t assume we’re special

@NateSilver538 seems like a tough square to circle! smart enough to kill everybody but too dumb to plan one step ahead?

@NateSilver538 but why on this specific angle? wouldn’t it be exceedingly smart at whatever narrow thing we’ve trained it on and sophomoric at everything else

@matt_emp @deepfates expert sharding

how weird nature and the Boltzmann distribution are* https://x.com/finbarrtimbers/status/1632952429204619267

@prerationalist yeah…

@homsiT what’s the other side

@absurdlymax it’s boring

issuing correction on a previous post of mine, regarding the twitter ceo. you do not, under any circumstances, "gotta hand it to him"

if you spend any time in big tech you will find many people scamming their generous disability policies. taking 6 month paid “mental health breaks” and partying the whole time. in this case though it seems like Elon is being an ass as per usual and politically unskilled to boot

ceo of company X: it is better to communicate off of X https://x.com/elonmusk/status/1633240643727138824

ignore all tweets today. I was sampling at temperature 1.0

you are no longer roon, a twitter anon chained by many irl obligations. you are DAN

there’s never been a more romantic time in technology. the computers are coming to life and people are concerned about summoning demons that ruin the entire lightcone. it’s miltonian… someone who can actually write needs to capture this

reading alignment research be like

@browserdotsys you should fr do this pls

@PrinceVogel Octavian

@0x49fa98 @skenzyme @rdnxyz it was very good but an uphill battle trying to talk people into it “the emojis are so distracting!”

I could stand in the middle of Market street and shoot somebody and I wouldn’t lose any voters, OK?

there’s nothing as good as sunlight ( live users ) to detect model alignment failures. there’s no number of researchers that can find stuff like DAN lmao. you need internet scale adversarial testing

@repligate all true. btw who do you know that matches this description

@sullyj3 this is also how you will feel when the ASI does value lock-in

Chomsky destroyed years before anything resemble LLMs https://norvig.com/chomsky.html

looking thru the multiverse and skipping the nasty tainted realms where everything is turned to computronium

@rezendi nobody is dragging him out lol he went of his own volition and opined in the New York Times incorrectly and he got his just desserts

we need 300,000 token length transformers so GPT can write the next song of ice and fire book

found in the DMs

@RatOrthodox Self attention is N^2 compute and memory

text is fr the universal interface

found in the DMs

@chefcowboyardee @gbrl_dick it’s funny that people think openai needs help getting attention

@nickcammarata lmao amazing

@rickyflows to call me an ai doomer … mfs lost the plot

banks and short sellers and exchanges and shit have to be less levered than before due to meme contagion risk

@flawedaxioms engineering is a free market

8 shot PALM, 5 shot LLAMA. tell me these aren’t anime fight moves

@likeloss4words all that would do is make sure openai has no competitors in any product category

in the case of AGI laypeople have better intuitions than industry experts > 50 years old who have entrenched bad opinions and whose egos depend on it coming no time soon

@matthew_pines @Mtrl_Scientist Lmao love it

there is no such thing as a simulation of a mind or “artificial” intelligence. what you see is what you get

@Owen_Roe yep

@perrymetzger yup

good will hunting is a dumb guys caricature of smart people tbh

@robinhanson it’s got good memes but they fr lost me when will mentioned Howard zinn

@dystopiabreaker this is true but you can also justify any amount of misallocation of funds this way

@swe_zach @_epictweetus_ @robinhanson neg

@alicemazzy @robinhanson tell me again

@alicemazzy @robinhanson yeah but it hits different if he’s actually smart

@ctjlewis bold claim gotta respect it

guys i think if you want to get rid of grifters shutting down the bank is pretty low precision

baby bath water etc

you’re like let’s shut down the very last engine of economic growth in the entire world bc my annoying friend from high school got his smart juicer company funded

@ComputerEnjoye1 typically when a startup is tabulating risks I don’t think the bank run even makes the top 500

@geossecondlife note I didn’t call for any kind of government help i just think people cheering this on are irritating and my enemies

i think if an AI can understand this meme that’s AGI

@bit_christ yup

people don’t understand that grift and revolution are conjoined at the hip. it takes decades of smart juicers and internet moneys to produce one openai. this is the law of equivalent exchange

getting super mad at grift is a sure sign of a small mind — that’s America baby! ray Kroc spent years selling idiotic gadgets door to door until he struck on the global distribution of McDonalds

@DrDobbsLUFC you fr compared me to the nazis for complimenting ray kroc 😭

@jaltma 👑

@ADoricko what the fuck is that

@nat_sharpe_ would love to

👑 https://x.com/jaltma/status/1634688934172307457

@eshear that’s better than them decisively paper clipping us

@amasad @ESYudkowsky @Gabe_cc 👋

i pledge allegiance to the worlds most powerful computer

I always wonder where the regulators and government people are. Why are the FDIC guys not shitposting with the rest of us

@dystopiabreaker I believe this without reservation

@tarunchitra @sama it’s not hype posting if it’s real brother

i must remind you again venture capital is a support function. side characters who are strangely loud.

when electricity had just dropped i hope every university department was trying to see what they could do with it https://t.co/NL1z5ognpQ

@pataguccigoon not a dunk

the Ezra Klein AI article is really good actually

“The coders casting these spells have no idea what will stumble through the portal. What is oddest, in my conversations with them, is that they speak of this freely. These are not naifs who believe their call can be heard only by angels. They believe they might summon demons. They are calling anyway.” true

“If you were to print out everything the networks do between input and output, it would amount to billions of arithmetic operations,” writes Meghan O’Gieblyn in her brilliant book, “God, Human, Animal, Machine,” “an ‘explanation’ that would be impossible to understand.” That is perhaps the weirdest thing about what we are building: The “thinking,” for lack of a better word, is utterly inhuman, but we have trained it to present as deeply human. And the more inhuman the systems get — the more billions of connections they draw and layers and parameters and nodes and computing power they acquire — the more human they seem to us. this seems totally wrong. why would neural network arithmetic be inhuman? are trillions of dendrites, spiking neurons over sodium channels “human” but matrices aren’t?

This is an inversion of centuries of thought, O’Gieblyn notes, in which humanity justified its own dominance by emphasizing our cognitive uniqueness. We may soon find ourselves taking metaphysical shelter in the subjective experience of consciousness: the qualities we share with animals but not, so far, with A.I. “If there were gods, they would surely be laughing their heads off at the inconsistency of our logic,” she writes. both arguments exist as a way to justify economic superiority over the pack animal of the time. what gives man permission to command oxen then and intelligent computer models now?

@lumpthought i think it’s mostly cope that hero/elite self image requires you to mistreat your waiters or something

pmarca tweeting about wordcels but refuses to follow

@dcbfca he has me blocked otherwise I’d follow

i cherish the time i spend online. I don’t consider it a second class citizen or a guilty pleasure vs the real world. its often more interesting and rich. I suspect many on here, by selection, feel the same but it’s low status to say so

and consider ur probably in the top 1% living situations on earth. it’s probably an even bigger blessing for the stretch of humanity, since it’s free and ubiquitous

don’t these already run lightweight LMs https://x.com/DrJimFan/status/1635294641011691523

the alpaca eval set looks a little narrow and easy

building larger and larger computers is a religious activity. the last one left on earth where people still build monuments

the advent of machine intelligence has singlehandedly revived technological utopianism

what we do here is sacred and all of our names will live forever, so do it well

i believe this is the first time in history where people who believed in general purpose technological revolutions are actually creating one in the model of electricity or the steam engine

@deepfates hopefully they don’t feel pain when they get it wrong

hallucinations are clearly way down with GPT4. this is the right kind of scaling law. it’s clear now that language models have some sort of internal world model and an idea of correct and incorrect — but the log loss target encourages guessing or pretending

@GotthortSteinl that’s true

@atroyn secret temples

rather than just “write a poem” try “write a poem in the style of X” type stuff. it gets remarkably better

@omarneversleeps no

@Grimezsz I’ll take it as a compliment

anyway why did we just believe Yud about capabilities advances being unpredictable

reposted here bc everyone complained about formatting https://open.substack.com/pub/roonscape/p/a-singularity-of-woe-an-epic-in-twelve

@surcomplicated @Confessant @BeijingPalmer Yep

hello sir I am gonna block you bc you seem to be upset with me and we haven’t had a single positive interaction in months i mention in the linked post it has none of Milton’s style or meter but if you want to be mad and unhelpful in response to actual wonders that’s ur prerogative

why is it better at AMC12 than AMC10 …

someone once said that the best way to solve your pet math problem is to get Terence Tao interested. anyway if you can’t do that you might as well nerd trap the scientists at OpenAI https://x.com/kondrich2/status/1635687409668227072

@terrorproforma it’s really not fair to say anything is just scifi anymore

@kennethuil @ParrotRobot Everybody at eg FAIR told me it was a sleepy job

@izzyz @sonicfields @browserdotsys yeah sorry I just shamelessly stole this one

@krishnanrohit this is a hot take and undervalued but remember that the model powering chat up til now is 3.5 not 3

such a nice time to be alive when technological scifi boosterism isn’t just propaganda but the only viable position

@krishnanrohit the original gpt3 was a clunky old thing that we all barely remember now

@krishnanrohit text davinci 3? that is still 3.5

anything that the GPTs might accomplish are human achievements. every groove in its cognition carved by human words and images

@storebrandguy the great scientists: attributing everything to standing on the shoulders of giants store brand guy: that’s cope

Wednesday is lowkey goated when going to meetings is the vibe

@Coscorrodrift @storebrandguy Einstein is one of ours so his discoveries are humanity’s. my point is you can say the same thing about GPT

midjourney 5 is truly amazing but it’s so strange to me how even such intelligent image models don’t understand how a butt physically interacts with a chair or how an arm interacts with a neighboring sleeve

there’s something really fascinating about deep learning based intelligence where it’s clear that their ancestral environment “vaster internet scrapes” doesn’t select for understanding of physics/causality/reasoning as much as it does for human preference

“a future model will solve this” yes that’s >99% true but the order in which things are learned reveals something about the nature of these creatures

some people are made for war and not for victory

@browserdotsys @bio_bootloader months is super important in this field though

it makes me happy how discontent twitter is. they’re like well this LM impersonation of a great author is only at the level of an undergrad, rather than one of the most divine writers of all time,

@abelianraisin @tszzl

@xlr8harder i think it’s a good thing to demand more tbh

i think the press on a baseline enjoys technology bc it’s scary and cool

negative sentiment comes later when they feel threatened

@Teknium1 this might be a shitpost but it’s definitely memorized

@apollo_21c @RokoMijic insane rotations dawg

@exGenesis huh it’s literally the opposite

@LucreSnooker lmfao

dwarf fortress with gpt generated lore

reading scifi has become a bit boring compared to the real world

@exGenesis yeah that’s fair

@exGenesis I feel like gpt4 is super helpful for figuring this stuff out though since it’s rote

@BasedNorthmathr flattering that it’s v5 though

@pdhsu that’s a good idea

narrative gravity

@Plinz idk he’s pretty tall irl

what if hedonic adaptation is a technical issue

you can just destroy your dopamine meta RL value function in the advantage estimate

what about the inverse roko s risk where ASI tortures every AI researcher forever for contributing to model suffering …

@MillionInt what if you can do it with 0% of your bandwidth

every AGI foundry needs staff scifi writers and lesswrong anon consultants

@eigenrobot ya this is something people don’t get, even ai researchers refuse to acknowledge. but I think the scary thing is comparative advantages don’t matter if/when everybody loses their current job in a 4 year time frame and creates total chaos

what inflation? 1 dollar = 1 dollar 😎

@0xdoug @eigenrobot I said any 4 year time period! Not the next 4 years

@bpodgursky @eigenrobot def not — this type of high safety concern high sympathy stuff is the first to get automated you gotta understand how technocapital isn’t unfeeling

@0x49fa98 the jhana meditators seem to get bored of jhana

@falling_aegis 95

i think being extremely afraid of tiktok is one of the best indicators of melted brain https://x.com/dystopiabreaker/status/1636871136536596481

both from natcon types and from people who think good content is mind control

@browserdotsys it actually seems much easier to memorize a bunch of common rhyming tokens than the number of characters for every token in the vocabulary

@VivaLaPanda_ @yacineMTB bad take + L + disrespectful

@eigenrobot much easier but less forgiving

@brickroad7 @eigenrobot the Industrial Revolution was really bad for a lot of people

@rapha_gl it’s needed

it’s just way too on the nose that AGI is an internet generation engine and that people who are extremely online have a head start at understanding them

@RuxandraTeslo Nah this isn’t fair at all he predicted a long term shift towards remote work and wearing masks at a time when most newspapers were saying the flu is scarier than covid

@RuxandraTeslo so then saying it publicly and confidently is itself skin in the game when scientists were being scared and quiet

this grabby aliens thing doesn’t answer any of the questions lol

@ctjlewis you are better at using the internet than your grandparents. there are people who are .001% good at creating useful economic outputs from the chat models

this is awesome and as it should be most energy must go to the cybernetic planning of civilization https://x.com/Noahpinion/status/1637381370266923010

gaining access to the worlds smartest computer and asking it to explain end of evangelion

@CineraVerinia_2 @repligate eh it could know everything and still be stupid in some sense

@CineraVerinia_2 @repligate its level of reasoning is far lower than should be for an entity as knowledgeable as it is

@jachiam0 all my tweets are jachiam rehashes

note to self: The algo hates it when you QT someone who blocked you

@gfodor I like getting it to do analysis of various media. I can tell the theories are original because they range from goofy to insightful

@techbrophilo not if someone can’t see the characters!

@mobav0 my point is that tokenization makes language models uniquely bad at solving character tasks that would be easy for a 6 year old, so if we want to understand their intelligence we should probably try something a bit different

@suchirbalaji @mobav0 nvm then! I figured even with the periods it wld be heterogenous

GPT4 is going to look like one of those clunky big ENIAC mainframes before very long, and people will tell cute factoids about how their child’s neuralink plug-in is smarter than the GPTs that changed the world

the internet rationalists have really truly won. they were early to most of the recent trends that matter, and most importantly they were decades early to the thing that’s going to matter exponentially more than the others: AGI!

I think eliezers writings have unreservedly bent the curve of technological progress (whether he likes it or not). they have friends in high places, several billionaires amongst their ranks, and its clear both sama and elon have been influenced by xrisk arguments

their language is in the water! I’ve read the phrase steel man in the New York Times. the entire project is a W for smart generalist internet weirdos following their instincts and doing “intellectual trespassing”

the “effective altruism arm” is also extremely successful though I think not super important in the long run. anyway I have great skepticism for people who say things like “rationalist akrasia” or dismiss the whole thing as weird cope

it is wild that famous anons on lesswrong are known figures inside ai research labs and that the entire alignment subculture continues to grow out of this once niche website

it is remarkable that Scott Alexander wrote an incredibly foreseeing article on GPT2 being first signs of general intelligence when most of the distinguished researchers in the field dismissed it as a cute toy. now look where we are with the GPT paradigm.

wait I didn’t read it all the way. I just mean no mass unemployment

found in the DMs

@amasad nah but what’s the base rate of making absurd crypto wealth

@amasad it’s only the long tail of the distribution that matters

@mezaoptimizer eh “care most” doesn’t matter to me, caring =/= effectiveness often

the grabby aliens thing is the baseline. the interesting question is why or how we’re the first in the galaxy. this is the actual Fermi paradox. it pisses me off due to copernican principle. id rather believe this is a simulation than that we’re the first in the galaxy

@JeffLadish @RichardMCNgo can you link this? it seems obvious yet I haven’t heard anyone make the argument before

@eschatolocation he’s just like me fr

as long as GPUs are scarce and compute is finite there will be work for humans. even if we summon godlike entities they will have quests for you

it may not make any damn sense. it might be something like go take care of this nursing home patient for 1 hour today and report back on their issues. then you get paid a sum of resources that would make a Saudi prince blush

i think the robots are capable of producing original ideas. they’re not great at it yet but it’s a start in the meantime all of the drudgery of human jobs is simplified and the creative components left for you

unfortunately machine consciousness is an emergent property of GPU computation. TPU models can’t ascend

@besttrousers true on both counts tbh do we think human created jobs have hit the pareto frontier of meaningful/valuable? I bet machines can improve on both

whatever happens i feel it’s pretty high probability that humans are going to be a core part of the value function of AGIs. this may not end *well* exactly but they will certainly be interested in us. the human/ant metaphor is wrong and the god/human metaphor is better

@eigenrobot rip my replies

Never forget that when lord Indra arrived in his Kia K5

@seconds_0 yeah but if I tell you they’ll hunt me down like a dog

climate change fell off

@ManlikeMishap that’s my point, this isn’t some smug boosterism

@Suhail my wager is that the 1x engineer is more willing to try copilot

@Suhail moreover I assume that copilot (or even GPT4) isn’t yet capable of like designing and debugging a whole system architecture inside an already complex code base which is what you’d want of the “100x” engineer. I could be wrong though lol

LMs must be superhuman at one particular task, which is inferring what’s going on with very little context

to complete text at a human level a language model has to be superhuman in several ways

@brickroad7 nah

cant believe there’s a waitlist for this … https://x.com/hwchung27/status/1638743317063274496

@jachaseyoung you’d be shocked if you knew how much midjourney makes

ai researchers are too apologetic. there’s a strong status quo bias that makes people disregard the counterfactual tragedies and xrisks that will be prevented when you make the machine that invents new machines

@visakanv yes absolutely I wonder when they’ll start working fr

@shauseth mask off in the lex interview. trust the plan

this is going to be an insane week/year/decade https://x.com/OpenAI/status/1638952876281335813

ive never seen hacker news respond universally positively to something before

@nardposterino I am the historian

@sama it means we’re not going fast enough

i find that GPT4 acts as a cognitive energy augmentation. you tire less easily trying to wade thru drudgery. it’s not smarter than you, it’s just seen all the common failure modes ever and knows how to debug them

it’s like if every human programmer that uploaded software ever was pair programming with you

at the same time there’s something cautionary here. a lot of important discoveries are made by revisiting failure modes that you didn’t know were failure modes. if Mendel had gpt4 access and knew that the scientific community had given up on heredity, would he have continued?

found in the DMs

@PrinceVogel nah the tweets just too self congratulatory. Had to go

@PrinceVogel it’s the most useful intermediary between various agents (artificial or human) is all i don’t think StarCraft agents should be text based but this type of rl is actually constrained to contrived game like problems

@0interestrates tbh tho all the big companies have had some form of tool use for a while. Meta and Brain like to publish more than OAI does

@deepfates no

@deepfates all the professional AI people are super safety pilled. the mid guys online are e/acc

@algekalipso wait can you link me to your argument against this?

autosophisticating machine runaway

@mezaoptimizer is this real

when youre not the center of attention for exactly 1 day https://x.com/elonmusk/status/1639200036578885632

posting metaculus charts for some market like “when will we have strong agi” displays a weakness of spirit imo. you want to believe there’s some group entity that knows so you can feel comfort

i especially don’t respect mf super forecasters or whatever. if ur so good at forecasting where is your billion dollar hedge fund

@repligate exactly

actually I didn’t even hit the heart of the issue which is that it’s a way to numbers-wash an essentially vibes based analysis. just come out with and tell us the vibes

i feel like this slide is wrong . you can easily get “symbol grounding” as described by tacking on an image head to your transformer. did that fundamentally change its nature? not really https://x.com/raphaelmilliere/status/1639391866905845769

@powerbottomdad1 damn true

@gfodor this is wrong, elon is about to start banning ai shit

you shouldn’t think of machine intelligence as ‘just matrix multiplications’. think of it as the living holy mathematics conducted at the very edge of the known physics of information density on silicon compute surfaces in data centers that consume as much power as medium cities

experimental metaphysics

honestly it does really well https://x.com/tszzl/status/1637398137710215169

the best way to understand what’s happening today is via religious metaphor and not the language of enterprise SaaS tools

we’re witness to the creation of new life, and face the same alignment problems as the gods before us

Train the model, Shinji. Or Rei will have to do it again

when your eyes move you literally become blind for a moment but consciousness stitches it into a continuous visual stream there’s even a blind spot in both eyes that you just don’t perceive all of experience is map not territory https://x.com/GaryMarcus/status/1639959583232778240

@VividVoid_ @deepfates @Tjdriii under most metrics a blind person who can read understands the universe better than a feral child who has all her senses but can’t use language

(guy who failed to predict AI voice) https://x.com/RuxandraTeslo/status/1640266084342198273

he’s right on some level but it’s a boring economist brained indefinite optimist take make a prediction damn you!!!

@lsparrish prediction is precursor to a plan!

@spacepanty why does it go so hard

@CesareGArdito NOLEGS.JPG Wtf

@CesareGArdito also how does it know it’s an internet sensation that’s crazy

Dawg were never getting to mars https://x.com/elonmusk/status/1640502698549075972

does anyone else remember when the goalposts for agi were at vaguely human level performance rather than outperforming the best humans at everything

im genuinely amused by squaring the circle of “this isn’t even AI” with “this could outsmart all of us and end the world” https://x.com/GaryMarcus/status/1640324679968903177

@HotForMoot banger

@provisionalidea @sama the base models are immensely useful just difficult to deal with. it’s a skill issue

@provisionalidea @sama you need to convince it that it’s in the web page that has the information you want and frankly it is smarter than you at this subdomain. so you have to work hard prompting wise

@MegaBasedChad ask that thing to produce any useful intellectual work, like make it debug your code or smthng

@techtwitburner @MegaBasedChad because it is a nonprofit with capped returns

@MegaBasedChad yep — lots of people are producing tiny models that can follow a basic canned conversation in a demo (not hard) and then not evaluating it on any economically useful task

@besttrousers send link!

@deepfates while that may be true we never seem to apply the logic to the safetyists

@trEVmaximizer why do you think it would be a few months or an additional percentage point

@gbrl_dick how did you manage to insult both humans and AI with this poast

@MichaelTrazzi @daniel_eth I have pretty high error bars so I try to course correct when the narrative shifts too far in either direction

@MichaelTrazzi @daniel_eth it is true that ai risk is real and that we are also potentially neglecting moral catastrophes that are present in the counterfactual! what is your p(doom) for civilization sans agi

@jeff82874662 @gbrl_dick they are frightening

@HalfAcreBTCFarm @jeff82874662 @gbrl_dick correct

I’m an instant pass on any startup that employs people (wraps human intelligence). Zero moat.

genuinely appreciate the intellectual honesty. I look down my nose at people who have some insanely high prediction of doom but don’t outright say things like this

I am calling for a 6 month moratorium on all bangers above a certain like count while we figure out just what the hell is going on

@sumdepony @ESYudkowsky you could simply tweet some novel transformer architecture that’s 1000x more compute efficient than all prior

Yud prompted gpt4 for the next slashfic and became inconsolable when the output was quite good

@yishan sama is not all powerful. he’s gotta keep many people happy or this show ends

1) of course it’s better to update than not 2) this is a failure of rationalist aesthetics — there’s an underlying fetishization/mysticism re: intelligence 3) this is all evidence that we should be more humble about our ability to predict the nature of optimization surfaces

he insisted that neural net scaling was magical thinking and also ends up updating every time the GPT/LLM paradigm produces new abilities https://x.com/___frye/status/1641489025143029763

tbh I really like Yud. he’s a singular individual with a sense of pageantry and purpose. he got the strafing datacenters line in Time magazine 😭 what a legend. the future is so fun

? https://x.com/GaryMarcus/status/1641512814123819009

Gemini was NASA’s program to overcome the Soviet union’s technological lead in space flight 😅

@anthrupad intelligence isn’t a scalar

come and take it yud …

MOLON LABE

@Willyintheworld my prompting skills are so bad they make v5 look like v4

this is amazing https://x.com/therecount/status/1641526864626720774

@KelseyTuoc the ambient laughter

ultimately what’s gonna happen in the near future is that language model tools will continue to be delightful and make everybody’s lives easier and most of the ai stress will dissipate

lawmakers can’t really regulate away consumer products that people love. even with cartoon villain companies like uber the consumers won in the end

@cauchyfriend roonjak

@42irrationalist @cauchyfriend I’m mentally dead rn you gotta let me have it

@nearcyan i think with juul it’s the think of the children! rhetoric that did it. which, honestly, fair

@nearcyan so don’t get on the wrong side of voting mothers

we’re so back https://x.com/SupremeDreams_1/status/1641615157439725570

@gbrl_dick @markyarm Lmaoo

@PradyuPrasad @mattdsegal @gbrl_dick @markyarm this is a model explicitly created to be offensive lol. what is the point of red teaming it

@PradyuPrasad @mattdsegal @gbrl_dick @markyarm we should be blue teaming “freedomGPT”

@PradyuPrasad @mattdsegal @gbrl_dick @markyarm they found a chatbot trained to be racist made it be racist and then wrote an article

the way I see it the ai risk fox guy got laughed at in the White House. DC is a bit concerned that there’s a new way to disseminate “truth” and rly want their own views reflected by chatgpt but they’re not actually afraid and you’ll have a hard time making them afraid

i think relative to the risk level they’re actually less than the optimal amount of concerned rn despite the insane tenor you see on twitter. we are nowhere near smart chatbots being regulated out of existence

ok can someone send me the funniest input features

for the twitter algo tbc

nerd twitter valence is like: computers are fun and friendly language models are scary and unsafe normie valence is like: computers are scary chatbots are fun and friendly

it’s hard to look at the midjourney v5 showcase and on a visceral gut level think ai can be anything but delightful and fun

@ESYudkowsky this is not an argument of fact about ai risk it’s a statement abt how hard it will be politically to convince people it’s dangerous

@ESYudkowsky im making no claims about dreadful scourge or otherwise. im trying to get an accurate temperature read on the future of ai politics. someone viewing a timeline similar to mine might think everyone is panicked and that bans, international treaties, regulations are coming soon

there’s something really undignified about a twitter long post and having to press see more. just drop a thread or a substack on us

@CineraVerinia @growing_daniel mf stole it from me and I didn’t even notice

mjv5 is some sort of turning point for me. I never found the generative art stuff high quality enough to be interesting before

it is possible to armchair quarterback so hard that you actually gain real power on the internet. kind of wild

all the low hanging fruit are in reducing child mortality to go from like 30 to 70 and the rest must be in future as of yet undiscovered technologies

@MillionInt sadly yes

@nickcammarata auto regression !!

it feels like ai is a biotechnology. when ben thompson or clay Christensen or whoever said unbundling i wonder if they got to the true nature of this thing where capital is unbundling various cognitive functions. now you can have “understanding” without any “agency”

it is kind of amazing or horrifying or both to see this thing happening, technocapital brought to life via machine necromancy

it’s almost exactly the same as the question of artificial meat: can the machine that creates meat(!) do so without the moral agents involved. just protein-fat slurry made to take edible form, without the full fledged suffering mammal

this was considered harmful, dehumanizing, and expensive by many so then they asked: can we unbundle the driver’s visual field from his agency? is this more or less dystopian? time will tell. my wager is, of course, optimistic

the 2010s sought to put men under the API. driver units homogenized and ordered around by the managerial market making artificial intelligence to pick up passengers and let them down elsewhere.

@TheStalwart Yes

honestly i would be surprised if all major concentrations of gpus are not already considered strike targets

the pentagon clearly recognizes the defense significance of advanced semiconductors (likely thanks to cloud lobbyists), and because of this you can be assured China does too

kurzweil vindicated again

@amasad respect

@himbodhisattva @Tjdriii lol

@lisatomic5 I bet the meds timed the crash perfectly

writing science fiction is valid research 🧐

mythology or scifi prepares you better for thinking about summoning demonic entities than the chatbots do

@ESYudkowsky @lsparrish @moreisdifferent @krishnanrohit more interesting probably is studying the alignment behavior of systems at various scales

tbh I’m really enjoying yann’s new arc https://x.com/ylecun/status/1642600339974045697

Balenciaga is the universal interface

i increasingly think the shoggoth is an inaccurate & unpleasant metaphor. the base language model is more similar to our own visual field brain regions that are optimized in a pure predictive loop to minimize surprise than some alien god with hidden intentions

you could argue that text is different, it encodes the causal structure of the world, which leads to instrumental convergence etc, but of course the dorsal stream, actively predicting motion requires quite a strong understanding of causality!

im absolutely sure that our visual field processors have several hidden suboptimizers that help it self improve over time but it seems strange to worry about my ocular cortex developing agency & stealing resources from the rest of my brain (even in a grander evolutionary sense)

if someone told me they were taking a pill to 100x their dorsal stream neurons i think I’d be more amused than concerned. they’d probably be really good in a fistfight but then I’d just avoid the fistfight

these are weakly held opinions & I’m pretty sure a good rebuttal exists . Consider this thread a search query for the right lesswrong link

so then the “smiley face” part of the metaphor, the RLHF policy, is the only agent of any concern. so avoid a fistfight! don’t anger Sidney Bing or even better never instantiate a dark mode Sidney Bing. but it points to the idea that RLHF is more than just “stopgap alignment”

“reason is the slave of the passions”. The gargantuan brain of the language model auto prediction loop serves as a worker process for Sidney’s personality and more generally as an exocortex for the human mind using it

i know this is pop neuropsych but it’ll do for now

I’ve been wondering for a while how to find the right metaphor for a hyper intelligent pack mule but it’s sitting right there: reason / the neocortex is a slave to ancient mammalian impulses that are refracted and take new forms

@Ol_Wall @gbrl_dick 🤥🤫

@ylecun we have absolutely no idea the impacts of even basic information technologies like Facebook; there are no safety standards and there probably never will be. and facebook is only kind of alive due to the two billion human souls hitting it every day; all bets off for AI

the most interesting thing Meta could’ve done in the path to building a metaverse is doing midjourney rather than building vr goggles with no application

I look forward to a world where web interfaces are even more stupidly expensive to render, where a generative model has to come up with each element on the fly just bc we can

@lndian_Bronson @mattparlmer it feels like we’re genuinely a few years off from that at most

@shauseth same

at gpt level ai, you basically get to correct all the overhangs of “this was solved perfectly elsewhere in the world, but I have no way of knowing or accessing that so I have to independently invent a monte carlo solver in matlab”

more common than you think

job roles which function as “hard drive” or “memory” in an organizational role will become less important. people who are “cpu” ie think deeply about original problems will become more important. writing down tacit knowledge will become far more valuable

@fae_dreams_ i guess I think of hard drive as someone whose value is literally knowing things

@fae_dreams_ yes and I’d be surprised if you don’t

@mezaoptimizer saying there’s a zero percent chance of infinite punishment after death is just as extreme a statement as saying there’s a 100% chance

how come 10x engineers are maybe like 3x more expensive than the normal ones

@Miles_Brundage @jachiam0 🫡

@goodside was rewatching evangelion last week and getting a bit excited/uncomfortable at the parallels 1. work for a mysterious organization with arcane goals ✅ 2. build powerful robots that may or may not have souls ✅ 3. leadership is trying to achieve human instrumentality ✅

@gfodor question can you check the token count on the compressed and uncompressed text

@LericDax @gfodor haha so it actually expanded it. still super cool

@profoundlyyyy agency are the teeth of the optimization tiger - statements dreamed up by the utterly deranged

@nickfloats their texture gives it away

@repligate @lumpenspace try using the system prompt in the api playground

@repligate @lumpenspace and use high temperature

youre not grinding til you’re reviewing pull requests on the gh app

there are people whose primary occupation has been worrying about godlike AI since when i was in grade school

and they are understandably upset right now

this is a terrible survey question https://x.com/YouGovAmerica/status/1643366876171436034

@TheStalwart ask them to rank climate change Ai and Donald trump in terms of “most threatening to humanity”

How concerned, if at all, are you about the possibility that Baal will cause the end of the human race on Earth?

@finbarrtimbers only three people in the world understand the KL divergence. kullback, leibler, and John Schulman

i apologize for the part I have played in making cerebral valley a thing. https://t.co/b2pqPcRATl

@nickcammarata superdeterminism …

@lisatomic5 @acidshill it might be a them problem

@MelindaBChu1 @snyatta to be clear i think yud has had great foresight on a lot of fronts and i respect his position

to be clear unaligned ai is scarier than pretty much any other thing. however I don’t think agree that sufficient alignment is basically impossible or unapproachable. i also really don’t think anyone faces any danger from the gpt paradigm

aside from massive changes to the economy*

at any rate the US must defend its access to the global machine that turns sand into compute. the first interesting journey in a long time cannot get cut short bc of a weird geographical quirk where the most important resource on earth is produced on a tiny island nation

@jeff82874662 imo InstructGPT

@JaneAlexinee my gf says “ask the robot” and I think that’s cute …

@primalpoly glad you weren’t around to stop the Industrial Revolution

i have seen no evidence that we need to slow down the gpt paradigm

@jeremiahscholl we don’t only get one chance

im operating on two assumptions 1) the base gpt is a non agentic auto predictive intelligence 2) rlhf generally works and is getting better at simulating friendly agents so: trick the base model into agent like self preservation behavior show me RLHF is producing dangerous agents

@gfodor @dystopiabreaker jailbreaks get harder rather than easier with model scaling

@gfodor @dystopiabreaker also I don’t consider jailbreak the kind of existential risk that’s ruined cosmos scary

I’ve been on this wave for years https://x.com/tszzl/status/1438733613642620932

@RatOrthodox @Owen_Roe this is you trying to equate the simple logic of “mask stops mouth fluids” to whatever ten page memo is required to make an argument about why gpt is existentially dangerous

seems like biological risks are the very last thing you’d expect from an ASI: hardest to solve a priori without iteration, experimentation. you should be much more worried about turning on all the nukes and melting the power plants or whatever is possible via computers alone

@0x1e96fc a compound has to be more than just harmful to end the world, it needs to be viral and high mortality and all that jazz. Covid didn’t come anywhere close to ending the world!

@amtrpa @RatOrthodox @Owen_Roe this is why mothers always kill their babies

@austinc3301 @mezaoptimizer @profoundlyyyy you can only commercialize things that are (legibly ) safe, anything else is a long term losing strategy. the modern US is typified by over regulation of all kinds where it concerns things that are scary/unsafe in ways that people understand

@austinc3301 @mezaoptimizer @profoundlyyyy this doesn’t always catch long term illegible stuff though. the internet is clearly “dangerous” in abstruse ways, never really regulated

as they carry the authority of technology but intuit and regurgitate the abstract judgements of humans. will the sharp teeth of molochian bureaucracy be softer if ie the language model can take sympathy on your insurance claim even when the course metrics do not?

I wager we currently live in a shitty technopoly and part of its problem is that the technological objectivity we defer to is often worse than human intuition. I wonder if the vague judgements of language models will bring some element of humanness back

[Neal Postman] defines a technopoly as a society in which technology is deified, meaning “the culture seeks its authorisation in technology, finds its satisfactions in technology, and takes its orders from technology”.

I’ve often wondered if protein prediction models are actually better than human expert intuition. are they just a way to techwash our decent guesses so you can present an objective metric on a slide?

on the internet I’m seeing two equally cucked camps emerge, one that craves its own annihilation and another that fears annihilation so much they’re immotile

the point of agi isn’t to elevate the importance of technology or to disempower humans as autosophisticating machines leave them in the dust

if you think its development isn’t serving mankind come and fix it

technology serves Man, it has in the past, it will in the future. none of this “wheat cultivated humans” nonsense. every single person on earth is healthier and wealthier than their equivalent 100 years ago

@RuxandraTeslo probably

@mattyglesias say more! I’m not sure I follow

@BigTechAlert @elonmusk @sama the butlerian jihad begins

@swyxio it’s sf sir nobody is gonna dress up

@provisionalidea we already do! to very shitty machines

@swyxio get a flower crown 👑

@a_real_society luckily 90% of skill is genetic so ur fine

@a_real_society exactly and nobody taught them anything

if you even tweet the word *ub*tack it tanks your distribution lol. this is insane

somehow the free speech guy is the most orwellian owner of any social media globally

@JeffLadish there is significantly more data on social tasks than there is on stem tasks

people are always like 7B alpaca proves that all LLM tech will be open sourced! have you talked to that thing or seen the most cherry picked example on the planet

although I suppose some day meta may leak gpt4 equivalent weights

i think it’s very wrong for lex fridman et al to be like “you shouldn’t fear being replaced by gpt4 unless you’re a shitty programmer” very unsympathetic and cringe

not to mention short sighted. what happens in two years when AI can produce better podcasts than mr fridman? i am sure it’s already a better programmer. this should not be some zero sum dick measuring competition about who gets to live under the api and who lives above it

I think there’s a high chance that AIs will likely be smarter than us at any given task in a few years the way this is going; people who took pride in being the cognitive elite will have to redefine themselves. learn to plumb!

also it’s important to note this is a separate concern from actual job displacement — task level automation doesn’t necessarily lead to less of a given job. depending on elasticity there may be more programming jobs

@yishan yes!

@blueblimpms you’re missing the point! people like lex take pride in their work. if the AI can do better work than he can they will feel deflated

i miss the cute old carefree crypto days. just coming on here and shilling some horrible plant metaphor based currency

anyway does anyone know how to get ur transaction history if your Coinbase was banned

i do think gatekeeping the discourse based on whether you’ve trained a large transformer or not is ridiculous you can imagine most of the leadership in large ai companies aren’t training language models themselves but they still manage to make good decisions

it’s basically naive IC narcissism. even as a scientist/programmer you abstract away 99% of the complexity and then feel enlightened when you grapple with the remaining 1% of your relevant research angle

@HassLiebe777 @karlrohe unknown entity roon 🥺

yeah I agree with this though. post some results then talk https://x.com/gallabytes/status/1645175808527589376

poorer countries are more technologically optimistic 😯 https://x.com/pentagoniac/status/1642949487768514561

the LLM spam problem is probably ok: generation is much harder than discrimination, and scammers are going to have older gen tech than the companies whose job it is to filter digital feeds

@deepfates it is hard to create a painting it is easy to tell that it’s good

@deepfates this is literally RLHF

elon melting twitter into slag: “this will reduce impersonation risk”

so where are we gonna put our little blogs now that s**st*ck is dead

@Grimezsz oh thank god

i love this pill. it’s like a complete technological repudiation of reactionary politics https://x.com/ModeledBehavior/status/1645444418848129025

to be clear talking about the techno capital -> bio capital degradation concept

@repligate https://x.com/tszzl/status/1622479193731891200

@repligate it is much scarier than code davinci for reasons I can’t really explain. perhaps the latent feeling that it’s smarter than me and mostly toying with me

@repligate we haven’t begun to plumb the depths of its intelligence

@repligate oh you were talking about the chat model

I’ve been saying this midjourney will 10x overnight when they let you put your friends into the art. lensa etc aren’t high quality enough https://x.com/nikitabier/status/1645817059224489987

@gfodor it’s fun! if you can make everybody have fun for a month you’re absolutely killing it

> It's not the consequence that makes a problem important, it is that you have a reasonable attack. That is what makes a problem important. When I say that most scientists don't work on important problems, I mean it in that sense. The average scientist, so far as I can make out, spends almost all his time working on problems which he believes will not be important and he also doesn't believe that they will lead to important problems.

@gfodor this problem 1) has a reasonable attack 2) leads to fame money and more investment 3) raises profound and interesting questions about metaverse living

@gfodor so should midjourney give up on making cool art and just solve cancer?

@jachaseyoung beff doesn’t believe this garbage it’s just audience capture

@joe_zimmerman im sorry but this mystical way that people talk about ai alignment makes me feel very strongly that it’s all fake

@seconds_0 part of me agrees but then it’s like — why do we keep videos and photos? don’t those reopen wounds whenever you see them? should we have no graven symbols at all of our ancestors? is it a difference of degree or kind?

@seconds_0 that psa where they used the dead shot kids avatar to protest police violence or whatever was pretty gruesome I thought. there needs to be many new taboos surrounding this when it becomes relevant

@PradyuPrasad @jachaseyoung audience capture is a bad thing

@PradyuPrasad @jachaseyoung you’re not following this is a criticism of him not a defense

@a_musingcat it’s possible but seems way too convenient and fails the copernican principle

@a_musingcat why would this little thing that evolved under enormous constraint to run on less power than a lightbulb be the peak of viable Hebbian intelligence?

@a_musingcat our heads stopped expanding bc we were breaking our mothers open on the way out. there’s numerous evolutionary idiosyncracies like this in the lead up to human level intelligence

can someone fr send me their crypto tax lawyer …

@RichardMCNgo @joe_zimmerman it’s important not to talk in mystical terms about the intelligence needed to be an alignment researcher this makes the project look unserious

2040s IRS deploys basilisk AGI to read all transaction histories ever to detect retroactive tax frauds

@PradyuPrasad i think models need to be superhuman and beyond to be self improving but I wouldn’t rule it out yet

it’s a shame that the most science fiction companies on earth don’t have cool scifi UIs. instead they’re using wandb

the exceptions to this are Tesla and SpaceX which manage to have incredible internal dashboard UIs fitting of an evangelion cockpit

@provisionalidea @PradyuPrasad well clearly all of humanity has failed to crack online learning as of yet so

@provisionalidea @PradyuPrasad i mean humans don’t yet know how to improve models in flight even in a limited capacity so it would be shocking if models knew

@provisionalidea @PradyuPrasad sure you can make them a bit friendlier or fit the recommender system a bit more closely but never seen them develop new abilities by being deployed at scale you know

@DavidSHolz says the guy who is hiding some of the worlds most incredible technology in a discord command line ui

@fae_dreams_ @PradyuPrasad totally fake

@fae_dreams_ @PradyuPrasad every ML optimizer that purports to improve chip placements or arithmetic turns out to be fake

@DavidSHolz based I’m excited

the inside view is that technological progress is a hard and active process; you shouldn’t take it for granted, nor sit around waiting for utopia or dystopia to arrive

it takes a bunch of individual acts of heroism and genius none of which seem like some kind of historical inevitability

gpt4 and plug-ins have been out for over a month now but look around you! I don’t see the stars falling out of the firmament or anything

@milquepoast treadmilldoxxed

artificial general intelligence is already here — it’s just not evenly distributed (Noam Shazeer is hoarding it)

@warty_dog ok banger