Abstraction levels and tool evolution, versus bugs - Re: strangest systems I've sent email from

[Swift Griggs]

On Wed, 27 Apr 2016, Toby Thain wrote:
> Modern languages can indeed wipe out large classes of bugs (including 
> many of those that lead to vulnerabilities). But *every* advance in 
> abstraction does.

While I follow your thesis here, I would also point out that the reason 
that a lot of C programmers roll their eyes when folks start talking about 
abstraction is that generally increases code size and overhead. Those two 
mandates (get out of my way, but don't let me make mistakes) are tough to 
achieve at the same time. People want the features they find in scripting 
languages (no memory management, no hassling with pointers) but they don't 
want to pay for them. It's a matter of debate among experts as to how much 
fat abstraction adds, or the value of the abstraction itself but it's the 
primary reason why many C coders are dubious of too much abstraction. C, 
in my opinion balances the two in one of the least-ugly ways (not that 
it's all that glamorous, it just works). Those on the "maximum 
abstraction" bus have already arrived at their stop in C++ land or have 
moved on to RubyTown etc... (not slamming them, just saying - I do C++ 
too).

I stick with C because I don't want (much) more abstraction than it offers 
for the applications I write or maintain.

> I like Professor Benjamin Pierce's way of putting it: "Mechanical checks 
> of simple properties enormously improve software quality."

Well, I definitely agree with this in general. The question again is how 
much are you willing to spend to get them done? There is always the guy 
who will say "Oh gawd we have 6Ghz machines with 4TB of RAM now. Who 
cares." That will always be followed by the person who says "Wait, I still 
do embedded code on a Z80 with 4k of RAM!" One thing that I'd point out 
about C is that once you learn it, you can do both (some would say poorly, 
but not me).

> The virulence, level, and number, change. Just think of the change in 
> the nature and frequency of mechanically missed bugs going between: 
> assembler to C; C to Java; Java to Haskell; etc.

True. 

However, when I look at http://benchmarksgame.alioth.debian.org/ and other 
such comparisons, and compare those languages doing real work I can see 
what the cost is to do so. Maybe some would assert that all this safety 
can come for free. My response to that would be "Excellent. Please write 
us all a new compiler, then." For the record, I do know about things like 
gcc-ssp, propolice, stackguard, etc.. I've used most of those where 
needed.

> I'd rather be dealing with only the bugs that get through that sieve, 
> than deal with malloc/free bullshit or buffer overflows in C.

I write quite a bit of C. I don't claim for a second to be "good" ('cause 
I've met people who are *really* good). In my experience there are several 
things that you can bring to bear which will result in you hardly every 
chasing pointers or malloc() issues. Here's what I do to help myself (BTW, 
I'm sure you are all better coders. I'm just sharing my experience):

1. Early on I wrote myself a reference counter. There are all kinds of 
   easy ways to do this. Then you can catch yourself when you use a null 
   reference or let a loop run off the edge of a cliff. Once you feel 
   comfy, remove the #include'd wrappers. I've done the same thing using 
   LD_PRELOAD mechanisms to preempt things like malloc().

2. When in doubt I've used tools like ElectricFence and Valgrind to also 
   aid greatly in finding problems that might lead to buffer overflows or 
   string format exploits. I also use tools like "rats" to hunt down bugs. 

3. I started getting burned enough that I simply figured out what kind of 
   design patterns I was using that got me into trouble. Once I applied 
   myself to stopping that behavior, the issues I was having with memory 
   shell games basically went away. 

4. Kind of along with the philosophy you are talking about (perhaps) I do 
   check the livin' snot out of everything I get back from 
   non-deterministic operations. Some of these are just using my knowledge 
   of the code's actual logic path to do the checks at the *exact* spot 
   they are needed without some compiler trying to out-guess me or check 
   absolutely everything.

YMMV, but that's how I live with my dirty C programming self.  :-)

> Productivity, security, reliability, correctness all demand that we wipe 
> out as many tiers of bug as we can, with better/more high level tools... 
> imho of course...

That's what I hear from folks who love Haskell and Erlang. As I'm sure you 
know, those languages have some really interesting properties similar to 
the kind of philosophy. 

You guys are going to be the type of people who move the whole industry 
forward as you push innovative (or maybe old & innovative) ideas. Guys 
like me just try to get the code done as fast as possible before our jobs 
get offshored to India. :-/

-Swift