Alt account of @Badabinski

Just a sweaty nerd interested in software, home automation, emotional issues, and polite discourse about all of the above.

  • 0 Posts
  • 349 Comments
Joined 2 年前
cake
Cake day: 2024年6月9日

help-circle

  • I’ll say that given the way OpenAI and Anthropic have hideously overextended themselves (they have over a trillion dollars of financial commitments to companies like Oracle), it’s not impossible that the current crop of American LLM providers do just kinda… poof away. Traditional banks want nothing more to do with them, they’re getting majorly spooked. All that needs to happen is for private credit to lose confidence in them, which is already happening. When they’re out of cash, they’ll be on the hook for an absolute ridiculous amount of money and they’ll probably just get liquidated.

    I’m sure there will be new companies that pop up, but they’re going to have to charge 10x what Anthropic is making enterprise customers pay, since inference likely still isn’t profitable at the price Anthropic is charging.

    I don’t use LLMs as a knowledge base because if the problem is bad enough for me, I’m likely just grepping through kubernetes source code or something. That being said, I don’t necessarily have an issue with folks using an LLM that way as long as they fully understand exactly how bad it is at what it does. You’ll be fine if you lose access to LLMs, and that’s the number one thing in my book. Your friend? Not so much.

    (As a fun bonus to all of this, Oracle is very likely to die if OpenAI can’t meet its commitments. Either way, Larry Ellison will probably stop being a billionaire, since almost all of his wealth is in Oracle stock).


  • The speed and ease at which LLMs allow you to generate code is a bug, not a feature in my opinion. In my org, a group of 3 very junior engineers wrote a 5k line shell script for building k8s clusters according to our business specs and it’s fucking awful. The actual time to get it out the door was short, but now it’s basically impossible to change it without fucking up like 20 different things. The fucking thing will randomly quit because the shit ass LLM thinks set -e is a good thing to use, and it’s full of unused variables everywhere. I had to add a feature to it (which is how I learned of its existence), and I spent a miserable week just reading the entire fucking thing so I could ensure that my change wouldn’t cause an oil refinery in the North Sea to explode due to a butterfly-effect series of bullshit.

    The frustration and toil you feel as a software dev is a feature. If something is making you mad and is taking forever to write, that’s a sign you probably need to change your approach. If you’re using an LLM to write a bunch of boilerplate, why not just eliminate the boilerplate or like, make a factory to spit out a bunch of it or something? Your discomfort is a powerful tool and you are not best served by ignoring it. Those junior devs would have written something much better if they had been forced to experience the true toil and suffering of writing a 5k line shell script.











  • Building a jet doesn’t require over a trillion dollars of capex, and selling jets is profitable. There’s solid evidence that inference isn’t profitable, and the AI labs need inference to be extremely profitable if they’re going to meet their absolutely ludicrous contractual performance targets. Oracle is expecting hundreds of billions of dollars from OpenAI by like 2030. That shit is not happening.


  • The segmented caching request thing is… weird. I worked for a company that developed a caching proxy and it very much did not work that way. Like, random access in a caching system is usually kinda bad and you should try to avoid it. Like, our proxy manually controlled the disk (it wasn’t a mounted filesystem) so it could constantly sweep the head across the disk and cue up reads and writes optimally. This gets much harder when things are fragmented as fuck.

    If the concern was about what would happen with multiple connections for the same cache miss, then the caching proxy should just combine the client-side connections into a single upstream one. You can still cache the first part of the response if your upstream connection gets terminated and then restart it from that point.



  • Yeah, it’s so fucking frustrating. I felt conflicted writing this because I don’t want to reduce anyone’s resistance to the garbage being pushed by the big corpos. We should be saying “no” as strongly as possible at every encroachment. I just also don’t want essential research to also take the blow. A lot of environmental research benefits from satellite imagery, and anything we can do to glean more information from that is a good thing.

    Damned if you do, damned if you don’t. You can’t really expect the average person to learn the distinction between the good and the bad here. You can try to educate folks, but people already have enough shit on their plates as it is.


  • This is why “AI” is such a shit term. This is not a general purpose generative model, which is what you (and me) should (and very clearly do) dislike.

    This is a model that is designed to operate on a very specific set of data and extract information from it. It was created by people at the University of Cambridge, not one of the big shitty companies. It’s not something that you run all the time, it’s something you only run when gathering data for research purposes. The model was trained on truly freely available data. No nonconsensual large-scale scraping was used to train this model, so it’s free of the ethical concerns typically associated with “AI”. Since it’s something a research group would run by themselves on a single (albeit very powerful) machine, it has very modest power requirements.

    Models like this have been around for at least 15 years in the research space, and they don’t deserve your ire. It’s one of the truly good uses of ML.

    If you want more details on the system, it’s all open source and can be found here: https://github.com/ucam-eo/tessera

    EDIT: Please don’t take this as me trying to defend LLMs and image generators. I fucking hate LLMs and image generators. People at my workplace have described me as “the anti-AI guy” because I really am. I think almost all of the ML products made by OpenAI, Anthropic, and others are unethical and also just shit.



  • I gave it no advice, and all I wanted it to do was generate a script to tell me the file type of the newest file in the current directory. It was a very trivial piece of code. Each time it generated something I disliked, I told it “don’t do this, reference this guide for the correct thing to do,” or “don’t do that, do it in such a way that X happens.” It was like 20 lines of bash in the end.

    I was expecting it to write me a bash script because that’s the example that everyone, without fail, says will work well. “I just used Claude to write a little throwaway script to move some files around” were the exact words a colleague used.

    Bash is a shitty, unsafe language. I don’t write large programs in it. I expect “throwaway scripts” to still be written in a way that defends against all of the innumerable shitass foot guns present in the language. Claude was incapable of doing this in a reasonable time frame.

    I also dislike the Python and Go it generated, while we’re at it. It produces overly verbose, overly documented, poorly performing code. It was also fucking dog shit at referencing runbooks and documentation in a local folder when I was on call and responding to alerts.

    It sounds like you’re quite partial to Claude, and I hope it’s been a very good and helpful tool for you. I did not find it to be particularly helpful for me. It was very good at putting me in a sour mood, however.