Portable assembler? - Re: Undefined behaviour in C - Re: Dennis Ritchie has died

On 10/14/2011 11:49 AM, Dave Caroline wrote: I sure hope so. Two jobs ago, I had to deal with a financial that had half their customers on one node of a VCS cluster, and have on the other running the same application. That application, in Java, had a few hundred thousand threads. No, I'm not kidding. This thing interfaced with MQ and of course had one thread per queue. Each client had something like 10 threads total, some MQ listeners, some to send results back to the client, some to do various other things, and in QA, for each client, there was a complete fictional counterparty, complete with its own set of threads. Of course, there were also pools of threads for database connectivity too. We couldn't throw more memory at the problem since the JVM at the time had a limit of 2GB. The only thing to do was to chop the application up into different VMs. Which of course would fail in their own interesting ways. Of course the incompetent programmers there decided to not use MQ for these JVMs to talk to each other - they used something MQ-like that the JVM had built in and spoke over http. Why? I've no idea, but this bridge went down quite often and required bouncing also. You can image what a clusterfuck that was in terms of both performance and debugability. There were monthly out of memory crashes as some run away thread didn't release memory. These were usually caused by running out of database threads. Whenever the db portion of the platform ran out of threads, for some reason the app decided that it should spawn of a thread for every incoming request. Or maybe that's what it normally did and I didn't notice it because when things were working that thread shut itself down after the request was processed. One very famous problem at that place was solved when they actually hired the one and single competent java programmer. Of course, he has since left there too. The problem had to do with padding strings with spaces in some report so that it would be exactly X bytes wide. It turns out that if you do it in a loop and append a space to the end, java creates X-len copies of that string, and each one has to be garbage collected, and of course was insanley slow. I think he simply invoked Java's version of printf to solve this. :-D I don't play programmer in my day jobs, though I can code in C and a few other languages. My role at that place, and in the current one is sysadmin. As incompetent programmers are apt to, they go and blame the database, or the network, or Veritas Cluster Server, or Solaris, or the hardware, anything but their own code. A lot of my time was spent *proving* that these other things were infact functioning as they should and that the programmers needed to re-examine their code. I probably spent more time looking at the stderr and stdout of the JVMs than the clowns that wrote that crappy code. Because this thing was so crappy and so likely to crash, it ran in production with the debug level in log4j fairly high, which produced gigabyte sized logs every hour, and of course slowed down the application by several orders of magnitude (of course the SAN got really slow due to so much crap being written to disk, and the programmers were blaming the SAN for the slowness of their app!) One of the big projects before I left there, was for the good programmers to go through the miles of code and to disable logging in places it was no longer needed. :-) A few days ago, I posted a message here about creeping layers of abstractions. The above is what you get when that's taken to absurd levels. What drives this? Corporations that want replaceable cheap cogs, that is one programmer to be identical to another so they can be replaced by cheaper ones, either in the same market, or outsourced overseas. They love java because everything is handed to them on a silver platter and they don't have to learn how linked lists or hash tables or quick sort or anything else actually work. They just say new foo bar and use that with no consideration to the performance implications. When it breaks, they've no idea why, or how, let alone how to go about fixing it. Had these guys tried to write this kind of thing in pure C it would have probably never compiled, or would have segfaulted every two seconds, thus pointing out their failures. But because it was Java, and Java is easy to learn, the company felt compelled to hire young guys who barely went through some Java 101 class and threw them at a financial app that supposedly processed billions of dollars per year. If you're a young guy looking at learning to code for a living, I highly recommend you learn what corporations consider esoteric languages instead, and either find a place that welcomes them, or start your own company. Learn C, and assembly then either smalltalk or lisp. I'm not saying that this is the state of the art in Java or other high level languages, but rather, because it's easy to learn, it allows this kind of incompetence to be acceptable. I'm glad I don't work there anymore, and instead work in sane place for and with bright people with a clue.