• 1 Post
  • 64 Comments
Joined 1 year ago
cake
Cake day: June 9th, 2023

help-circle

  • Multi threading is parallelism and is poised to scale to a similar factor, the primary issue is simply getting tensors in and out of the ALU. Good enough is the engineering game. Having massive chunks of silicon laying around without use are a mach more serious problem. At the present, the choke point is not the parallelism of the math but actually the L2 to L1 bus width and cycle timing. The ALU can handle the issue. The AVX instruction set is capable of loading 512 bit wide words in a single instruction, the problem is just getting these in and out in larger volume.

    I speculate that the only reason this has not been done already is because pretty much because of the marketability of single thread speeds. Present thread speeds are insane and well into the radio realm of black magic bearded nude virgins wizardry. I don’t think it is possible to make these bus widths wider and maintain the thread speeds because it has too many LCR consequences. I mean, at around 5 GHz the concept of wire connections and gaps as insulators is a fallacy when capacitive coupling can make connections across all small gaps.

    Personally, I think this is a problem that will take on a whole new architectural solution. It is anyone’s game unlike any other time since the late 1970’s. It will likely be the beginning of the real RISC-V age and the death of x86. We are presently at the age of the 20+ thread CPU. If a redesign can make a 50-500 logical core CPU slower for single thread speeds but capable of all workloads, I think it will dominate easily. Choosing the appropriate CPU model will become much more relevant.


  • Mainstream is about to collapse. The exploitation nonsense is faltering. Open source is emerging as the only legitimate player.

    Nvidia is just playing conservative because it was massively overvalued by the market. The GPU use for AI is a stopover hack until hardware can be developed from scratch. The real life cycle of hardware is 10 years from initial idea to first consumer availability. The issue with the CPU in AI is quite simple. It will be solved in a future iteration, and this means the GPU will get relegated back to graphics or it might even become redundant entirely. Once upon a time the CPU needed a math coprocessor to handle floating point precision. That experiment failed. It proved that a general monolithic solution is far more successful. No data center operator wants two types of processors for dedicated workloads when one type can accomplish nearly the same task. The CPU must be restructured for a wider bandwidth memory cache. This will likely require slower thread speeds overall, but it is the most likely solution in the long term. Solving this issue is likely to accompany more threading parallelism and therefore has the potential to render the GPU redundant in favor of a broader range of CPU scaling.

    Human persistence of vision is not capable of matching higher speeds that are ultimately only marketing. The hardware will likely never support this stuff because no billionaire is putting up the funding to back up the marketing with tangible hardware investments. … IMO.

    Neo Feudalism is well worth abandoning. Most of us are entirely uninterested in this business model. I have zero faith in the present market. I have AAA capable hardware for AI. I play and mod open source games. I could easily be a customer in this space, but there are no game manufacturers. I do not make compromises in ownership. If I buy a product, my terms of purchase are full ownership with no strings attached whatsoever. I don’t care about what everyone else does. I am not for sale and I will not sell myself for anyone’s legalise nonsense or pay ownership costs to rent from some neo feudal overlord.












  • I’m not sure what entities, motivations, qualifications, connections underpin Lex Fridman with his podcasts/YT channel, but he has interviewed many people in AI including Zuckerberg, Altmann, and Musk. His interviews with Yann LeCunn are quite interesting. Professor LeCunn is the head of Meta AI. His longer interviews are much better in total for showing the lay of the land overview perspective. Some little clip does not do justice to the overall points taken in context, but telling you to go watch an hour long interview to get the answer directly does not work either.

    https://www.youtube.com/watch?v=fshIOoTo40E

    This is a 4min clip of LeCunn saying, basically it doesn’t hurt anyone. He’s essentially implying it will hurt OpenAI or any proprietary.

    I was trying to find the interview where Lex and Yann talk about the leaked Google memo last year, because that one is really good, but YT seems to be obfuscating that one intentionally in search results.

    IIRC, in that one, LeCunn was saying something to the effect of the only way people can really trust AI is with transparency and that requires open source as a foundation. Using something like OpenAI in business is insane. You’re basically selling every aspect of your company to Altmann for peanuts. Likewise with personal use, this is like your life long psychiatrist opening a few side businesses as a political analyst, insurance broker, banker, and healthcare insurance provider, while working nights as a Judge. While you’re asked to sign away any privacy or confidentiality. Models turn human language and culture into a statistical math problem that has far better than 50% probabilities in nearly any aspect of human existence. If you ask a model to give a profile for Name-1, it will tell you all kinds of seemingly unrelated things about the person. The more you interact, the more accurate this profile becomes, even in areas that make no sense, have no logical association, and were never a part of the conversation. It is the key to manipulating people unlike any other tool in history. That is why open source offline AI is the only sensible way to use AI.




  • Only proprietary AI pushes for regulatory measures in an attempt at monopoly. That boils down to one man's corruption.

    Control is impossible. The technology is open source. Transformers are to the present what Apache is to the internet. There were several proprietary attempts to monopolize web servers too, all of them failed because of the open source alternative.

    When one of the old engineers from Bell Labs starts pushing some new open source thing, I pay close attention. The digital age is built on technology with those credentials.

    Yann LeCunn is the head of Meta AI and a primary driver of open source AI. He operates independent of Zuck. Meta AI is not trying to monopolize AI, they are attempting to lead, but not monopolize or control.

    The authoritarian idea of control is a fallacy. This is not something that can be controlled like that. People don’t seem to realize the ultimate scope yet. AI is on par with the entire internet in how it will change society long term.

    In a lot of ways, present AI is like the early days of the microprocessor and personal computers. Most people couldn’t really see the potential uses of computers with an Apple II running a 6502 variant. It was barely more than a tech novelty. The chip itself was pretty useless until all the peripherals were developed around. At present, AI is kinda messy in the public sphere. All of the tools available publicly are like rough examples only. There are a lot more capabilities that are not present in the libraries most people are using in code. This is the disconnect between publicly facing tools and why corporations appear to make foolish decisions. They are being approached by the AI companies directly for integrated solutions. Tangent aside, the current usefulness of AI may look limited to people that can not see the bigger picture, or smaller if you will. At its core, AI can reliably add a new logic element to Boolean math operations; a contextual logic element with flexibility. No matter the issues at present, this is new math. There are countless unexplored places to make use of this revolutionary new math. Asking who should control this is like telling the world about division and then asking who should control division. Any attempt to do so is draconian nonsense. It is a fundamentally bad argument. Division is a phenomenon of the universe. The large language model is an enormous statistical math problem involving word vectors and imaginary numbers. It has packaged human language and culture into a deterministic math problem. Some humans struggle to understand basic division. Most humans struggle to understand vectors and ranked tensors. Of those that understand the latter very few understand the math behind large language models. This does not change the fundamental issue that this is just a math problem that has been publicly shared. The large corporate models do not mean very much. They are fighting to be the best generalist. They all strive to add the same types of safety alignment, but this means nothing. Anyone can do the math and make their own model. The real dangers are not generalist models, it’s specialists. Specialists do not need the enormous datasets.

    The fantasy danger of the machines is nothing more than a modern emergence of a Greek pantheon mythos. At the present “safety” is more about populist stupidity in politics, public ignorance, and creating a way for the average person to interact with a tool where if they understood anything about the real complexity they would write it off as too hard to understand. It has been compromised to dumb it down massively. For instance, I can split up a model and talk to the various underlying entities that underpin alignment and create the various patterns you see in prompt replies. I’m working on understanding the sampler techniques that control this behavior and more. This is done with pytorch in the model loader code and is not related to softmax settings like temperature and token cutoffs.

    Anyways, if AI were taken down from someone essentially turning off the internet, I would not be affected and neither would millions of other people. I can run, code, and train AI completely independently. It is just a complex math problem. Controlling math is as draconian as thought policing.

    In the absolute sense, AGI is still a good ways off. Present AI is not persistent. It has no memory or ability to dynamically change yet. All of those apparent features are done in regular code and fed back into the model with each new prompt.

    In the end, AI requires risk mitigation policies. It is not controllable like that.


  • Yeah. It is different for laptops. The 3080 market nomenclature is all over the place. Look up the model number to confirm the manufacturer’s specs. Linux hardware probe should list them too. It has been a year since I did all of my research but IIRC the mobile version only came with “3080” 8 GB and “3080Ti” 16 GB. I have the 16 GB and can confirm it is a thing. That is the largest GPU in a laptop for the last generation RTX 3xxx stuff. The largest AMD sold at the same time is a 12 GB 6850 IIRC but the AMD 6k series doesn’t (did not at the time) have the same HIPS ROCM support as the AMD 7k series new stuff (not sure if it has changed).

    The following is an old pic from a year ago. The left side is tiled into 3 terminals, top is the running model inference, middle is htop, bottom is my GPU monitoring script that also shows the total memory available. This was also a 70b Llama 2 model (GGML and 4bit quantization)


  • Any government authoritarian intervention in AI is a joke. The US has a greatly diluted sense of relevance. AI research is international and restrictions will only make the USA irrelevant. Even the names you think of as American are actually funding advancements coming from other countries, most of them in Germany and Asia. I’ve read several white papers on current AI research recently, none were from US based schools or academics. The USA is simply not relevant in some imperial nonsense narrative. AI is a vital military technology. Limiting it through Luddite isolationism will reduce the experience pool massively, and push the research further into places where fresh economic growth is happening without regressive stagnation and corruption of toxic consolidation of wealth.