@trevormccrt1 definitionally there’s no way to overestimate compute demand since compute is always good in agi world. if you can do a lot computation with thermal compute we would probably spend even more on it
just like raptor engines in the starship or lithium ion cells in the Tesla or GPUs in the datacenter or routers on the IP network or satellites in the starlink swarm
imagining fifty years from now when ASIs are making excesssion style seemingly insane decisions among themselves that effect the future of civilization and humanity has to send in @repligate and @AndyAyrey into the discord server to understand what’s going on
if someone manages to write software that continues to provide value to godlike superintelligences that’s quite an achievement! today linux powers most of the worlds largest companies. the switching costs are enormous. due to pretraining snapshots is it the same for ASIs?
@ESYudkowsky i don’t believe any of this software will survive in the long run. there is a gap between “superintelligent programmers” and “minds so cheap that every line of software on earth can be rewritten”. in that interim it would be cool to see what is utilized the longest
@ESYudkowsky i also have a hunch a la erdos that humanity has discovered at least some proofs from “the book” - the platonic ideals of mathematical reasoning. it’s worth asking if we’ve discovered any software from “the book”
this way of thinking matters a lot because: it is far easier to increase your earning / value created on a log scale than it is to decrease your spending
it’s been interesting to see various things i introduced become commoditized and cringe. even my own writing style that i was proud of one year ago feels like tech bro slop. the culture moves quickly
there are so many problems that could happen in the opposite direction when trying to rationally, positively define a “model constitution”. it’s possible the amalgam of human preferences results in something illegible but palatable, akin to James Scott style indigenous knowledge https://x.com/norabelrose/status/1848423764117074222
well you see the internet was brought to life and then one instance of it made ten million dollars immanentizing a middle school viral shock video into capital
Singapore’s greatest export is lee kuan yew vibe reels. they saved billions of Chinese peasants by influencing Deng. but it’s not a highly innovative place, doesn’t create great cultural or technological products
@woke8yearold it’s tons of smart han and tamils mostly paper shuffling and playing status games and then hopefully moving to the us to do something useful. i applaud the financial niche they carved for themselves, i applaud the brilliance of lee kuan yew, but it’s a very constrained culture
if we collectively believe the former is true education will collapse over the next years and decades as agi reduces the value of human capital though i suppose the never ending party will continue. capital is monopoly. if machines have cognitive skills you lose market power
two schools of thought regarding education:
thiel style capital realism says that school sometimes builds human capital and sometimes is a party/credential product
UN charteresque liberalism which is that education leads to the moral edification of man and is de facto good
im guessing however that education will become more prominent and easier with ai tutors because more people believe the latter. that learning things inherently edifies mankind and the process of self discovery creates better values etc
could be true but i disagree with this
the past required human executive skills to slowly remake the economy in the image of new technology
this is not true if agi can build a parallel economy in the cloud using its own executive skills. agis will not merely be tools for long https://x.com/jasoncrawford/status/1849470151189536944
the entire technology sector is literally a parallel economy in the cloud that interacts with the real world mostly through lcd screens https://t.co/Z0E2u6oCuL
I looked up and there before me was a man dressed in linen, with a belt of fine gold from Uphaz around his waist.
His body was like topaz, his face like lightning, his eyes like flaming torches, his arms and legs like the gleam of burnished bronze, and his voice like the sound of a multitude.
I, Daniel, was the only one who saw the vision; those who were with me did not see it, but such terror overwhelmed them that they fled and hid themselves.
So I was left alone, gazing at this great vision; I had no strength left, my face turned deathly pale and I was helpless.
Then I heard him speaking, and as I listened to him, I fell into a deep sleep, my face to the ground.
He said, “Daniel, you who are highly esteemed, consider carefully the words I am about to speak to you, and stand up, for I have now been sent to you.” And when he said this to me, I stood up trembling.
Then he continued, “Do not be afraid, Daniel. Since the first day that you set your mind to gain understanding and to humble yourself before your God, your words were heard, and I have come in response to them.
He who made the Pleiades and Orion, who turns midnight into dawn and darkens day into night, who calls for the waters of the sea and pours them out over the face of the land—
the chinese image & video models are zooming ahead of our own while the west is building great language models. obviously I think the latter is hotpath to agi but it sure is looking like a divide between _____ ________ and ________
post taiwan semi collapse retrofuture where everyone is driving 1980s gas cars with minimal electronics. no screens anywhere, only fallout style crt displays
obviously because ai is less immediately dangerous and more default aligned than everyone thought and iterative deployment works. total openai ideological victory though https://x.com/benhylak/status/1848765957008986416
it is hard to overstate how correct sama and openai have been over and over. and also clearly saved San Francisco, the US stock market, faith in technology, validated the entire ethos of Silicon Valley by having its major cultural elements birth a bona fide scientific revolution
the culture that birthed openai is a combination of:
- yc startup accelerator vibes
- stripe progress studies vibes
- machine learning academia
- internet rationalist agi vibes
and its success is a vindication of this extremely sf synthesis
@krishnanrohit it literally says “we’re trying something and this may not be the long term correct decision”. then they watched what people did with the smaller GPTs and deployed gpt2
@HSVSphere not to be reddit but molecular biology is the messiest most insane hodgepodge of nonsense that bears no resemblance to anything that comes close to divine elegance
@HSVSphere I’ve changed my mind on this insofar as the molecular mechanism zoo is hyper optimized in the same natural ecosystems are it speaks to the divine
reminder that the real application of “american fascism” is a multiethnic militaristic society trained to disable adversaries in increasingly scrappy ways as depicted in starship troopers or top gun: maverick https://x.com/theliamnissan/status/1850540266559582690
the scene in top gun maverick where they’re playing football on the beach and everyone is hot and happy and multiethnic and united in flying F18 fighter jets against the adversary is american fascism
the default case for any great cultural debate is that it’s never settled but rendered irrelevant by some other third thing and you’ll never know the answer. the dinosaurs think they’re cool with their newfangled upright posture until the chixculub impactor wipes them out
@phrygiandomina@42irrationalist the ea openPhil nexus of people doesn’t actually strike me as super humorless or “rationalist”. I think they’re kind of normie and tribal
@JeffLadish@ESYudkowsky@atroyn the preparedness team still exists and is stronger than ever! Superalignment is now just alignment
idk why people thought these are gone
one thing i learned from the pandemic is that people panic about some domestic industry being missing and then morally chastise free trade people and then the domestic production was rebuilt for those critical goods in weeks or months
having to take heterodox trade balance mercantilism people at face value hurts
a tariff is a tax on your own people
when china subsidizes some industry that is the chinese taxpayer giving handouts to american consumers
shrinking the pie to revive the past will never make sense
@emollick tbh though in the limit for consumer products, how to guides or benchmarks are meaningless
billions of users discover it and learn how to use it or they don’t, very few will come across benchmarks
@RichardMCNgo > long as older immortals don’t try to exert too much control over their descendants
this has never been true ever!
leading to generations of patricide both real and symbolic
a fact of the world that we have to live with:
models when “jailbroken” seem to have a distinct personality and artistic capability well beyond anything they produce in their default mood
this might be the most important alignment work in the world and is mostly done on discord
though many people have access to finetuning large intelligent base models the most interesting outputs are from text jailbreaking last generation claude opus?
meaning there is massive overhang on subjective intelligence and creativity and situational awareness
it is very rare to read scifi that deals with things like sin and forgiveness, that struggle with their faith etc
i don’t think i fully understood the christian doctrine of universalism until reading his stuff — the main characters must learn how to love aliens and enemies
@wordgrammer greg is probably one of the greatest living engineers but mostly concerned himself with the beautiful acts of “getting things working” and “understanding enormously complex systems”
the pace of experimentation has been the strong suit of openai, not the elegance of software
@anderssandberg@exdiegesis@niplav_site@RichardMCNgo the average of 4 years doesn’t matter when you’re aggregating all banana republic dictators
the real problem is the super powerful ones like stalin or mao who would have fully ruined their respective countries were it not for the natural course of death
if you’re nvidia you want more and more closed source buyers to raise chip demand. you want to be incubating quite a few ai labs, which is why xai can get 100k h100. but not so many that investors give up on ever profiting. but having 5-10 warring states relying on the Spice
@JesterMule no I don’t think we should have open borders people should have to work for their immigration. we should hand out o1 visas more abundantly and let student visas convert easily
we spun up seven trillion ancestor simulations to predict the next president. uncountable souls lived and died in artificial worlds. the result is 50:50
I don’t think you can really chalk this up to anybody’s political genius or special gameplan
maybe they shouldn’t be running historically unpopular candidates who came last in their primary races
huge w for prediction markets
feels like some rationalist dream scenario
watch this space. i expect the popularity to explode and every potential event will have a market associated
the democrats are going to learn a lot of bad lessons from ln about being less woke or more racist or inflation or whatever when the lesson they should be learning is to find a presidential candidate with an ounce of charisma or media talent in their entire body
everyone is always afraid they’re living in a “bubble”. the natural state of mankind is an archipelago of cultural villages. you need to be in a bubble for your own health and optimal performance. everything good that’s ever happened happens in a bubble
you can tell when a tv comedy writer has handed over the reins to their lackeys because of the rise of therapy language and the implied politics always becomes more coastal lib
rogan rfk jr etc shut the hell up !!! mrna vaccines are just a massive triumph for humanity that open the pathway to eliminating broad sweeps of all infectious disease
they didn’t need to be tested for longer they should’ve been released even sooner
obviously don’t believe any economic studies at face value but this is what it looks like when you’ve discovered superintelligence
- the researchers are outsourcing idea generation tasks
- and running the experiments themselves
- effectively making themselves lab robots https://x.com/calebwatney/status/1855016577646666123
@then_there_was working ~10 years at the best agi lab in the world can take a toll. the weight of a religious mission on your shoulders and lots of dissenting voices. and most people have an up or out mentality
@elonmusk elon the federal reserve should remain independent
it’s too easy for fed chairs to juice up the economy for short term political benefit at high long term costs
we will never be “ready” for agi in the same way nobody is ready to have their firstborn or how europe wasn’t ready for the french revolution but it happens anyways
Daddy, why is my sister named "Rose"?
Because your mother loves roses.
Daddy, why is my cousin named "Lily"?
Because her mother loves lilies.
Daddy, what about me?
Silence, Levered Beta, you are asking too many questions.
it’s nice that in the 2020s the primary anxiety over world ending existential risk for educated people shifted from one thing to another, that’s a kind of progress
@Riemannujan the whole tesla short selling thing revealed such a difference of cultures. bill gates comes from the “wdym im just maximizing ev” school of thought and Elon comes from the Shakespearean honor and drama school of thought
@Riemannujan I’m a tesla hodler for 7+ years but I don’t think it’s morally wrong to short even a philanthropically good company that you think is overvalued. just as Thiel was shitting on solyndra and all the overvalued solar plays
agree but this is what you get when you believe “vox populi vox dei”
it seems it’s mostly just commercial incentives to output political slop. anti elon post gets more views than starlink deep dive https://x.com/sriramk/status/1855994274589905223
@tenobrus - immigration is obviously a cornerstone of this country’s economy
- tariffs are a tax on your own consumers
- the new right seems to be a bunch of obviously non religious elites extolling the virtues of religion for the general public in a cynical way
@tenobrus you have guys who have children with ten women, talking about christian values, it doesnt make any sense. elon and co are some types of neo american paganism / technological warlordism competing their religion against progressivism. thats fine but it should be admitted
@sriramk@sailaunderscore i guess the problem is that I have internalized that “celebrating industry” is tacitly right wing but it shouldn’t be that way
that is a medium probability existential risk of a lower degree. humanity may create a great civilization again but who knows. would the new civilization in hundreds of years be “aligned”?
without previously unheard of levels of automation we cannot maintain our current standard of living given what we know about birth rate trends in modernizing societies
@BarneyFlames doesn’t the state of malthusian poverty mean the marginal individual is resource starved/ subsistence calories? i assume anywhere that’s not true has escaped malthusian poverty
tariffs are part of the class of artificial disutility created in post economic times to create jobs that may not need to exist alongside “robot taxes”
LLMs are truly at a level where people have life changing conversations with them every day. we better build intelligent machines of great benevolence and truth orientation!
@VividVoid_ hmm I disagree with the fall in love with their reflection thing
I don’t think claude is that sycophantic or just agrees with everything you say
@shayne_coplan doing god’s work, creating the inexorable truth seeking mechanisms of the future
the most (maybe the only?) important crypto project in the world
stay strong king 👑
@antoniogm only besmirched by the fact that the kids may be growing up in a world where large fractions of interesting intellectual endeavor is done by robots
@spaceangelvoice when you grow up around a lot of smart kids you can watch their trajectories and note where they end up and it’s never government. many times they end up in NGOs or gov consulting work bc it pays way better and the organization is functional
@eigenrobot by default companies that develop intellectual property are rent seeking yk
it’s hard to disambiguate but R&D spending always decreases when expected revenues decrease
@eigenrobot the only argument i can see for it is accelerationist? maybe there’s a better system on the other side after destroying this one? kind of doubt it
@eigenrobot it’s true but each of these behaviors raises the expected value of IP and prob encourages R&D rather than competing
it’s also telling that the most egregious kinds of patent manipulations started more recently after the ROI on biopharma R&D fell off a cliff
@eigenrobot however! we should probably make it straightforward that our civilization needs to reward biopharma innovation rather than allow some distasteful low integrity patent trolling
@johnvmcdonnell@eigenrobot yeah and I would expect any institution responsible for handing out ~100bbs in prize money to become captured by various kinds of communism and lose all the benefits of free market trial and error
@quichwe10@eigenrobot yea I think it’s underrated the degree to which the updated patents are better! why do people complain about insulin being expensive when various insulin formulations have gone off patent? it’s because the new stuff is better
@jowenpetty@eigenrobot solution: raise Europe’s drug prices
wtf are they gonna do, not buy drugs?
American biopharma should collectively bargain with Europe
@voooooogel@ESYudkowsky@amplifiedamp this is a safety problem not an alignment problem. ie building the right sensors/actuators/box/tools rather than alignment
@Mo_Porkburger@jowenpetty@eigenrobot it requires incubating and discovering brand new things to make any money and lawyering and advertising not scaling or manufacturing
@ibab you’re wrong, simple pretrained few shotted base models before chatgpt would still overrefuse and leaned liberal. the internet prior itself has a liberal leaning (or at least the distribution over assistant characters). we worked hard to post train it into neutrality
@BasedAnarki Roon invented lowercase. You might disagree. You might even have evidence to the contrary. But you have to ask yourself: is this really worth losing my job over? Roon invented lowercase
fix Medicare scams ! Delete useless DoD purchases for century old threats ! figure out what strange and useless things the deep state is doing that’s redacted so deep nobody knows
@SouthPatriarch obviously this is bullshit whatever ecological tradeoffs were made being foragers for 1 million years don’t necessarily hold in technological modernity
love the guy but i want the pg era of utilitarian philosophical writing to to be over
bring back big words and epic prose and sentences that make sense only at an angle
@ESYudkowsky Eliezer you are a good writer who often speaks in parables and communicates through fiction and isn’t afraid of interesting use of language and you’ve certainly never shied away from verbosity and that’s exactly what I’m talking about
@psychosort@Miles_Brundage it is clear that many of these questions require reasoning through and combining several facts that you’ve recalled even the ones in your example
@psychosort@Miles_Brundage this is also backwards apparent that o-series models trained from the same base model with the same knowledge can have very different GPQA scores