Fixed point financial data versus floating point - Re: Spreadsheets (was Microsoft flamage)
24 Oct
2011
24 Oct
'11
9:25 a.m.
Decimal math (hardware supported) is used heavily in financial processing with IBM COBOL.
No loss of precision because the type is base 10. BCD is very similar to what IBM calls
"packed decimal".
------Original Message------
From: Fred Cisin
Sender: cctalk-bounces at classiccmp.org
To: General Discussion: On-Topic and Off-Topic Posts
ReplyTo: General Discussion: On-Topic and Off-Topic Posts
Subject: Re: Fixed point financial data versus floating point - Re: Spreadsheets (was
Microsoft flamage)
Sent: 24 Oct 2011 16:07
On Sun, 23 Oct 2011, Chuck Guzis wrote:
I don't care if the radix is 2, 10 or 13. What is
normally termed
"floating point" is a slight misnomer.
Agreed.
What SHOULD we call it? (Something short and [over]simpler than "IEEE
32 bit floating point representation standard")
It's interesting that while some vendors seem to
be intent on
dropping decimal support on CPUs, there are others proposing decimal
co-processors-- and the POWER6 and 7 CPUs have always had decimal
floating point capabilities.
I will admit to sometimes using some of the BCD instructions in the 80x86
family, such as AAM, DAA, etc. 'course, other than my sales tax program,
I usually use them for stuff other than BCD.
--
Grumpy Ol' Fred cisin at xenosoft.com
24 Oct
24 Oct
10:24 a.m.
On 24 Oct 2011 at 16:25, Vintage Coder wrote:
Decimal math (hardware supported) is used heavily in
financial
processing with IBM COBOL. No loss of precision because the type is
base 10. BCD is very similar to what IBM calls "packed decimal".
Only COMP-3 was packed decimal; COMP-1 and COMP-2 are single- and
double-precision real/floating/binary exponential. All others are
usually internally integers (scaled where a point is specified)
unless DISPLAY, which is kept as decimal character.
Some COBOLs have other data types such as BINARY-DOUBLE, COMP-5 and
explicit specifications such as PACKED-DECIMAL.
--Chuck
10:58 a.m.
I am aware of the various options, all I was saying is that packed decimal is heavily used
for financial calculations. I believe you have to specify COMP-4 to get fixed point
binary, the default USAGE is DISPLAY (for IBM) which is zoned decimal. There are no
instructions to do math on zoned (display) fields, so conversions are done by the compiler
if those fields are used in calculations.
IBM has COMP-5 but I'm not clear from the doc what it does, although it may be more
meaningful on Z since the ADDs support longer binary operands. Prior to Z it seems COMP-5
and COMP-4 would have been the same on IBM implementations.
-----Original Message-----
From: "Chuck Guzis" <cclist at sydex.com>
Sender: cctalk-bounces at classiccmp.org
Date: Mon, 24 Oct 2011 10:24:44
To: General Discussion: On-Topic and Off-Topic Posts<cctalk at classiccmp.org>
Reply-To: "General Discussion: On-Topic and Off-Topic Posts"
<cctalk at classiccmp.org>
Subject: Re: Fixed point financial data versus floating point - Re:
Spreadsheets (was Microsoft flamage)
On 24 Oct 2011 at 16:25, Vintage Coder wrote:
Decimal math (hardware supported) is used heavily in
financial
processing with IBM COBOL. No loss of precision because the type is
base 10. BCD is very similar to what IBM calls "packed decimal".
Only COMP-3 was packed decimal; COMP-1 and COMP-2 are single- and
double-precision real/floating/binary exponential. All others are
usually internally integers (scaled where a point is specified)
unless DISPLAY, which is kept as decimal character.
Some COBOLs have other data types such as BINARY-DOUBLE, COMP-5 and
explicit specifications such as PACKED-DECIMAL.
--Chuck
11:38 a.m.
On 24 Oct 2011 at 17:58, Vintage Coder wrote:
I am aware of the various options, all I was saying is
that packed
decimal is heavily used for financial calculations. I believe you have
to specify COMP-4 to get fixed point binary, the default USAGE is
DISPLAY (for IBM) which is zoned decimal. There are no instructions to
do math on zoned (display) fields, so conversions are done by the
compiler if those fields are used in calculations.
Who needs zoned instructions? We did display arithmetic the CDC
machines in display code using ordinary binary primitives (see my
post on Univac "tricks"). And we didn't have character
addressability.
During my tenure at CDC, the 6000 series machines running COBOL
consistently beat the IBM high-end iron on benchmarks, in spite of
the IBM sales team telling customers that CDC made FORTRAN machines.
A fast CPU is generally a fast CPU for any language. You don't need
lots of specialized instructions.
--Chuck
25 Oct
25 Oct
1:16 a.m.
On 2011-10-24 19:24, Chuck Guzis wrote:
On 24 Oct 2011 at 16:25, Vintage Coder wrote:
A long time ago, I was instructed that the "native" S/370 (integer) data
format was S9(4) COMP and that it should be used whenever possible for
fastest performance (counters, indices, etc.). I've now worked on
non-IBM iron long enough that when I go back to the 360 family, I'll
have to resort to the manuals to see if that still holds true and what
formats are preferred these days on the fancy Z hardware.
Mind you, these things are no longer considered important, nowadays; "we
got plenty of storage and processing power to spare", so just use
whatever comes to mind and let the conversions be done by the compiler.
It's not until the weekend batches start overlapping the Monday morning
online processing that people show up at my desk asking how to squeeze
their jobs back into the batch window. Even then, a re-order of some
WHERE clauses in SQL statements usually shaves about 50% off the
processing time.
Some of these guys seem genuinely surprised that the order in which
statements are placed can actually have influence on performance.
Imagine that! And it's not just the programmers; competent DBA's that
know more than to up the storage quotum when a table runs out of space
are few and far between, too.
Programming with performance in mind is a vanishing art and an unpopular
skill (in practice, that is; on paper, everybody pays lip service to it)
these days. The solution is almost invariably to throw new hardware at
the problem. Can't blame them, really. I mean, if I were in charge and a
programmer told me, "give me three years and I'll have it running twice
as fast on the current hardware," I'm pretty sure my reaction wouldn't
be to give the go-ahead on that one.
.tsooJ
Decimal math (hardware supported) is used heavily
in financial
processing with IBM COBOL. No loss of precision because the type is
base 10. BCD is very similar to what IBM calls "packed decimal".
Only COMP-3 was packed decimal; COMP-1 and COMP-2 are single- and
double-precision real/floating/binary exponential. All others are
usually internally integers (scaled where a point is specified)
unless DISPLAY, which is kept as decimal character.
3:39 a.m.
New subject: Fixed point financial data versus floating point
From: Joost van de Griek <gyorpb at gmail.com>
A long time ago, I was instructed that the
"native" S/370 (integer) data
format was S9(4) COMP and that it should be used whenever possible for
fastest performance (counters, indices, etc.).
I don't believe this was ever true on System 370 and as far as I know, not
on S/360 either. In IBM COBOL S9(4) COMP is a signed, halfword binary
integer. There are instructions to do halfword adds, subtracts, etc. but
the fullword version is probably faster or cheaper since the halfword
instructions produce a fullword result. You may as well get the full
operand width and use the fullword instructions. More than performance, the
issue was probably storage in those days. A fullword is twice as big as a
halfword and when you have less than a meg of core every couple of bytes
counts. Halfword counters have a pretty small range so you have to make sure
whatever you're counting will fit or that rollover doesn't hurt your code.
Counters in COBOL are usually non-negative and indices should be
non-negative unless you like taking chances with unsupported features. I
know at least one guy who did this. He exceeded the permissible size of a
table so he defined another one before it and used negative indexing (or
subscripting) to point to the previous table in storage even thought the
code referred to the second! Anyway in the counter or index case you would
normally be better off with an unsigned 9(4) COMP where 0 to 65535 was
enough range or 9(8) COMP if you need 0...(2^32)-1. Again this is all for
counters and indices as you said, for money it's inappropriate.
I've now worked on non-IBM iron long enough that
when I go back to the
360 family, I'll have to resort to the manuals to see if that still holds
true and what formats are preferred these days on the fancy Z hardware.
Nothing has changed, there are data types for all occasions and the only
thing added was the IEEE floating point with either XA or ESA, long ago.
Use packed decimal for money, fixed point binary for counters and indexes,
zoned for display fields, and hex or binary floating point if you like to
go head to head with Chuck on number crunching benchmarks!
Programming with performance in mind is a vanishing
art and an unpopular
skill (in practice, that is; on paper, everybody pays lip service to it)
these days. The solution is almost invariably to throw new hardware at
the problem. Can't blame them, really. I mean, if I were in charge and a
programmer told me, "give me three years and I'll have it running twice
as fast on the current hardware," I'm pretty sure my reaction wouldn't be
to give the go-ahead on that one.
In practice choosing the right approach and coding sanely seems to produce
satisfactory results in critical pieces of code. I'm talking about the
commercial space because that's where you and I work and this may not apply
to embedded or other environments where the pipelining and
microoptimizations are better understood than where we work. Nobody should
think performance isn't a deadly serious thing in our environment, as you
said, the batch window is a factor and system availability is no joke
because of the fines, loss of revenue, and loss of customers that occur
when production is not up on time. Sometimes that is all the time. Companies
are willing to spend millions of dollars on one product to measure their
throughput or improve their transaction rate.
4:17 a.m.
New subject: Performance-oriented development in the commercial computing world (was: Fixed point financial data versus floating point)
On 2011-10-25 12:39, vintagecoder at aol.com wrote:
True enough. The problem, as I see it, is that too few programmers pay
attention to performance issue at programming time, and fixing a large
system after the fact, while sometimes painfully obviously possible,
becomes too large of an effort to undertake. Buying faster hardware will
solve the problem quicker, at equal or lesser cost.
It offends my programming heart to see code checked in that clearly
could benefit from some simple performance tuning and optimisation in
choosing storage formats. But "this is the tested version, changing it
will involve another complete test cycle" and there is always a deadline
to meet.
We migrated to new hardware at $WORK recently, and of course the first
few tests were promising: "wow, this thing really flies!" Well, just
wait until we've migrated out bloated database full of dead wood to it,
and all that great processing power will go towards making the damn
thing barely usable. In other words, looking on the bright side, "oh,
we'll soon bring this monster to its knees, too!" Heh. at least we have
room to grow, now. Lots of space for more Itanium boards in the machine,
provided HP manage to stay in business...
.tsooJ
More than performance, the issue was probably storage
in those days.
C'mon, I'm not /that/ old! ;-)
Programming
with performance in mind is a vanishing art and an unpopular
skill (in practice, that is; on paper, everybody pays lip service to it)
these days. The solution is almost invariably to throw new hardware at
the problem. Can't blame them, really. I mean, if I were in charge and a
programmer told me, "give me three years and I'll have it running twice
as fast on the current hardware," I'm pretty sure my reaction wouldn't be
to give the go-ahead on that one.
In practice choosing the right approach and coding sanely seems to produce
satisfactory results in critical pieces of code. I'm talking about the
commercial space because that's where you and I work and this may not apply
to embedded or other environments where the pipelining and
microoptimizations are better understood than where we work. Nobody should
think performance isn't a deadly serious thing in our environment, as you
said, the batch window is a factor and system availability is no joke
because of the fines, loss of revenue, and loss of customers that occur
when production is not up on time. Sometimes that is all the time. Companies
are willing to spend millions of dollars on one product to measure their
throughput or improve their transaction rate. 
7:50 a.m.
New subject: Performance-oriented development in the commercial computing world (was: Fixed point financial data versus floating point)
More than
performance, the issue was probably storage in those days.
C'mon, I'm not
/that/ old! ;-) True enough. The problem, as I see it, is that too few
programmers
pay attention to performance issue at programming time, [...]
We migrated to new hardware [...] "wow, this
thing really flies!"
Well, just wait [...] and all that great processing power will go
towards making the damn thing barely usable. [...]
I co-run a mud. One of the other admins (who works for Intel,
amusingly) has a connect message set saying
"The most amazing achievement of the computer software industry is its
continuing cancellation of the steady and staggering gains made by the
computer hardware industry." - Henry Petroski
/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTML mouse at rodents-montreal.org
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
9:10 a.m.
New subject: Performance-oriented development in the commercial computing world (was: Fixed point financial data versus floating point)
On 25 Oct 2011 at 10:50, Mouse wrote:
"The most amazing achievement of the computer
software industry is its
continuing cancellation of the steady and staggering gains made by the
computer hardware industry." - Henry Petroski
I remember an Intel engineer summarizing this more succinctly
somewhat like this:
"We work hard to build chips with amazing performance and the
software people piss it all away."
--Chuck
10:15 a.m.
New subject: Performance-oriented development in the commercial computing world (was: Fixed point financial data versus floating point)
On Tue, Oct 25, 2011 at 5:10 PM, Chuck Guzis <cclist at sydex.com> wrote:
On 25 Oct 2011 at 10:50, Mouse wrote:
An easy cure is give developers slow machines with limited memory and
slow disk subsystems.
I do that to myself. I find it a good way to discover the "right way" TM
Dave Caroline
"The most amazing achievement of the
computer software industry is its
continuing cancellation of the steady and staggering gains made by the
computer hardware industry." ?- Henry Petroski
I remember an Intel engineer summarizing this more succinctly
somewhat like this:
"We work hard to build chips with amazing performance and the
software people piss it all away."
--Chuck
12:01 p.m.
New subject: Performance-oriented development in the commercial computing world (was: Fixed point financial data versus floating point)
On 25 Oct 2011 at 18:15, Dave Caroline wrote:
An easy cure is give developers slow machines with
limited memory and
slow disk subsystems. I do that to myself. I find it a good way to
discover the "right way" TM
I still am in awe at the speed of the average personal computer.
When programming the analysis software for the PAL "cloning" project,
I was concerned about running nested loops inside of an outer loop
that iterated 65K times (PAL16L8 chips for example, have some tricky
aspects, such as tristate outputs and feedback lines), remembering
how long a simple 65K decrement-and-jump sequence took on a 4MHz Z80
(about a second, if memory serves).
Of course, it didn't matter. The analysis finishes near-
instantaneously. Running a truth table with 16K entries through
Espresso also completes in almost no time at all.
I'm clearly of the "old' mindset.
--Chuck
12:51 p.m.
New subject: Performance-oriented development in the commercial computing world (was: Fixed point financial data versus floating point)
On 25 Oct 2011 at 18:15, Dave Caroline wrote:
An easy cure is give developers slow machines with
limited memory and
slow disk subsystems. I do that to myself. I find it a good way to
discover the "right way" TM
This is a recipe for expensive software. I've seen this idea floated
verbally more than once, but any company that ever cared about
performance or memory footprint (and there are plenty) never did
this, and for good reason. It makes your software ridiculously
expensive because you cripple the productivity of every person
working on it. You're better off developing automated benchmarks
against your software that test the performance or footprint sizes
and having those benchmarks fail with a noisy report as soon as
someone steps across the "that's too much" threshold.
If most of the people in the world aren't doing what you think is
smart, then perhaps you should look deeper into the situation before
declaring the rest of those people idiots. Maybe they know something
you don't.
--
"The Direct3D Graphics Pipeline" -- DirectX 9 draft available for download
<http://legalizeadulthood.wordpress.com/the-direct3d-graphics-pipeline/>
Legalize Adulthood! <http://legalizeadulthood.wordpress.com>
1:02 p.m.
New subject: Performance-oriented development in the commercial computing world
On 10/25/2011 03:51 PM, Richard wrote:
On 25 Oct 2011 at 18:15, Dave Caroline wrote:
Perhaps, but "the rest of the world" is producing operating systems
that require billions of bytes of memory and billions of clock cycles
per second just to BOOT.
I respectfully submit that they don't, in fact, know as much as they
think they do, but they want fast machines on their desks.
-Dave
--
Dave McGuire
New Kensington, PA
An easy cure is give developers slow machines
with limited memory and
slow disk subsystems. I do that to myself. I find it a good way to
discover the "right way" TM
This is a recipe for expensive software. I've seen this idea floated
verbally more than once, but any company that ever cared about
performance or memory footprint (and there are plenty) never did
this, and for good reason. It makes your software ridiculously
expensive because you cripple the productivity of every person
working on it. You're better off developing automated benchmarks
against your software that test the performance or footprint sizes
and having those benchmarks fail with a noisy report as soon as
someone steps across the "that's too much" threshold.
If most of the people in the world aren't doing what you think is
smart, then perhaps you should look deeper into the situation before
declaring the rest of those people idiots. Maybe they know something
you don't. 
2:08 p.m.
New subject: Performance-oriented development in the commercial computing world
----- Original Message -----
From: "Dave McGuire" <mcguire at neurotica.com>
To: "General Discussion: On-Topic and Off-Topic Posts"
<cctalk at classiccmp.org>
Sent: Tuesday, October 25, 2011 4:02 PM
Subject: Re: Performance-oriented development in the commercial computing
world
Computers and software are no different then every other piece of
electronics sold. You basically take the same model add a new feature and
maybe some small styling changes and make people upgrade every few years for
no real reason. If they don't follow along you start changing connectors or
software requirements to make them upgrade. There is a whole advertising
segment meant to make people feel like scum if they are not up to the
current standards of the masses.
If I had to guess the "rest of the world" is using 5+ year old machines and
software they paid little to nothing for while western (and some up and
coming Asian ones) countries pay to keep Intel, Microsoft, and the rest
rich.
Anyway I would suspect most programmers are counting on processors having
more cores (and systems having more RAM) in the time it takes for them to
complete a software project and get it out the door. Both hardware and
software makers need to keep pace with each other just to make money. If
Microsoft was still selling Windows 2000 the hardware side would be pretty
much out of business. If Intel can't keep coming up with faster processors
(or more cores) then software sales would plummet. As it is only a few
companies are raking in the money while others just tread water. Nobody is
jumping into the software/hardware selling field where margins are a few
percent at best, all efforts seem to be in advertising and data mining.
One of the reasons I like old hardware and software is in the single tasking
nature of it all (yes I know you can multitask but things just seemed more
like task switching). Most old software does one thing, there is no massive
integration with the net or every other software package. If anything you
might even be more productive since you don't have all those annoying
popups, it is just more relaxing.
The reason all this changed is because of market saturation. Everyone who
you could sell a computer to had one (or more) by the year 2000, after that
you needed to speed up upgrades. Sooner or later this constant upgrade cycle
will fall apart and many more companies will go bankrupt until we are stuck
using machines and software for a much longer time period and the few
programmers left are stuck trying to relearn optimization techniques again.
If most of the
people in the world aren't doing what you think is
smart, then perhaps you should look deeper into the situation before
declaring the rest of those people idiots. Maybe they know something
you don't.
Perhaps, but "the rest of the world" is producing operating systems that
require billions of bytes of memory and billions of clock cycles per
second just to BOOT.
I respectfully submit that they don't, in fact, know as much as they
think they do, but they want fast machines on their desks.
-Dave
2:41 p.m.
New subject: Performance-oriented development in the commercial computing world
On 25 Oct 2011 at 17:08, TeoZ wrote:
The reason all this changed is because of market
saturation. Everyone
who you could sell a computer to had one (or more) by the year 2000,
after that you needed to speed up upgrades. Sooner or later this
constant upgrade cycle will fall apart and many more companies will go
bankrupt until we are stuck using machines and software for a much
longer time period and the few programmers left are stuck trying to
relearn optimization techniques again.
I figure that the next Great Leap Forward in Sales Engineering will
be proliferation of "Sorry, this requires a 64 bit CPU" software.
There's quite a bit out there already, but one can still ignore a lot
of it...
--Chuck
6:01 p.m.
New subject: Performance-oriented development in the commercial computing world
In article <4EA6CAA9.19211.14EE95D at cclist.sydex.com>,
"Chuck Guzis" <cclist at sydex.com> writes:
I figure that the next Great Leap Forward in Sales
Engineering will
be proliferation of "Sorry, this requires a 64 bit CPU" software.
There's quite a bit out there already, but one can still ignore a lot
of it...
I've worked on applications that genuinely need the 64-bit OS and
compilation model in order to keep customers productive. Granted, it
isn't notepad, but to stick your head in the sand and pretend that
all of the demand for 64-bit systems is purely a sales gimmick is
just stupid.
--
"The Direct3D Graphics Pipeline" -- DirectX 9 draft available for download
<http://legalizeadulthood.wordpress.com/the-direct3d-graphics-pipeline/>
Legalize Adulthood! <http://legalizeadulthood.wordpress.com>
9:43 p.m.
New subject: Performance-oriented development in the commercial computing world
On 25 Oct 2011 at 19:01, Richard wrote:
I've worked on applications that genuinely need
the 64-bit OS and
compilation model in order to keep customers productive. Granted, it
isn't notepad, but to stick your head in the sand and pretend that all
of the demand for 64-bit systems is purely a sales gimmick is just
stupid.
No, I didn't say that. Right now most ordinary people can use a 32-
bit OS and applications and have a good match for their net
connectivity speed (particularly in the US). But I see 64-bit
hardware and OS being peddled to my neighbors who share the same
1.5Mb/sec down/768Kb/sec upload speeds. Hell, I don't expect my
connect speed to change enough to challenge the throughput of a
10BaseT network within my lifetime.
How long before Microsoft drops 32-bit support and the apps all
follow? Not long, I'll wager.
--Chuck
(It's funny--we were supposed to get universal 40Mb/sec net
connectivity by 2000 as part of the Bellcore settlement. Eh, what's
200 billion dollars, anyway? We're no. 21! We're no. 21!)
6 p.m.
New subject: Performance-oriented development in the commercial computing world
In article <BCA884FA5E22402B87487D0A971B0DDB at dell8300>,
"TeoZ" <teoz at neo.rr.com> writes:
Anyway I would suspect most programmers are counting
on processors having
more cores (and systems having more RAM) in the time it takes for them to
complete a software project and get it out the door.
While some people work like this, it isn't most. I regularly
interact with a large community of developers outside my daily
employment, so I have data to back up my assertions, not just an
opinion I formed because 30 years ago I coded small character based
applications in 16K of RAM.
--
"The Direct3D Graphics Pipeline" -- DirectX 9 draft available for download
<http://legalizeadulthood.wordpress.com/the-direct3d-graphics-pipeline/>
Legalize Adulthood! <http://legalizeadulthood.wordpress.com>
6:32 p.m.
New subject: Performance-oriented development in the commercial computing world
----- Original Message -----
From: "Richard" <legalize at xmission.com>
To: "cctalk" <cctalk at classiccmp.org>
Sent: Tuesday, October 25, 2011 9:00 PM
Subject: Re: Performance-oriented development in the commercial computing
world
In article <BCA884FA5E22402B87487D0A971B0DDB at dell8300>,
"TeoZ" <teoz at neo.rr.com> writes:
I would guess it depends on what you are doing.
Console developers know that their target platform is not moving at all, so
if they expect the next games to be any better or faster they HAVE to
optimize or wait for the next console (and lose sales). Kind of sucks how
some console games ported to faster desktop machines run like crap because
they were designed for a console to begin with. How much optimizing can you
do these days when the game you are building is using a game engine you
purchased which might run faster on a Geforce GPU then an ATI or vice versa?
I have no idea what embedded systems are like these days , they used to be
processor, storage, and RAM starved so they did need optimized but you also
knew exactly what the resources were.
If you are programming shareware utilities you might want the code to work
on a large range of machines and operating system versions so code size and
speed might be optimized to work on older slower gear but you also might not
be able to user tools that need specific OS revisions to work.
People who design the OS that need backward compatibility while needing new
features and eye candy probably expect more powerful systems to be available
and design the operating spec for this (Windows 7 was not designed for a
Pentium 1 for example).
Years ago when Intel started using MMX instructions in the Pentium 1 line
were developers smart in using that optimization in their code knowing that
most of the new machines (but not all) coming out will have it? Things have
changed quite a bit since the days of 8 bit computers where you knew exactly
what the resources and chipsets were AND your app would be the only thing
running.
Anyway I would suspect most programmers are
counting on processors having
more cores (and systems having more RAM) in the time it takes for them
to
complete a software project and get it out the door.
While some people work like this, it isn't most. I regularly
interact with a large community of developers outside my daily
employment, so I have data to back up my assertions, not just an
opinion I formed because 30 years ago I coded small character based
applications in 16K of RAM.
-- 
7:29 p.m.
New subject: Performance-oriented development in the commercial computing world
On Oct 25, 2011, at 9:32 PM, TeoZ wrote:
I have no idea what embedded systems are like these
days , they used to be processor, storage, and RAM starved so they did need optimized but
you also knew exactly what the resources were.
They still generally are processor, storage and RAM starved these days, but the spectrum
is a lot larger on the top end. We had a customer who insisted that the CPU doing the
housekeeping for an FPGA board had to be an 800MHz PowerPC with half a gig of RAM running
Linux. They insisted on this because they needed to keep up with about a 1KHz interrupt
rate and couldn't be bothered to learn how to use a proper embedded OS (or
pseudo-RTOS) which would have done the same on a little embedded ARM with a few hundred K
of RAM (or perhaps even tens, but it's hard to fit an IP stack in anything that small
and have room left for anything else).
A lot of these decisions are driven less by the engineers than the project managers who
were promoted out of engineering so that they could do less damage (protip: they
don't). All the engineers on this project (including most of the ones from the
customer) fought this tooth and nail.
Years ago when Intel started using MMX instructions in
the Pentium 1 line were developers smart in using that optimization in their code knowing
that most of the new machines (but not all) coming out will have it? Things have changed
quite a bit since the days of 8 bit computers where you knew exactly what the resources
and chipsets were AND your app would be the only thing running.
True, but MMX instructions (and their modern counterparts, the SSE branch) seldom make a
general-purpose difference. Most people for whom they will make a difference (game and
media programmers, particularly) typically are the types who will make use of them. On
x86, at least, pretty much all hardware since 2005 supports at least SSE3 except for the
barest-bones stuff (Atom, for example, is a mixed bag).
- Dave
9:45 p.m.
New subject: Performance-oriented development in the commercial computing world
On 25 Oct 2011 at 22:29, David Riley wrote:
On Oct 25, 2011, at 9:32 PM, TeoZ wrote:
The problem is that I'm not sure what "embedded" means any more. Is
a smartphone with a Cortex A15 4-core processor "embedded"? How
about an Android Pad?
The distinction is blurring pretty fast.
--Chuck
I have no idea what embedded systems are like
these days , they used
to be processor, storage, and RAM starved so they did need optimized
but you also knew exactly what the resources were.
They still generally are processor, storage and RAM starved these
days, but the spectrum is a lot larger on the top end.
10:26 p.m.
New subject: Performance-oriented development in the commercial computing world
On 10/26/2011 12:45 AM, Chuck Guzis wrote:
The problem is that I'm not sure what "embedded" means any more. Is
a smartphone with a Cortex A15 4-core processor "embedded"? How
about an Android Pad?
The distinction is blurring pretty fast.
Putting full-blown UNIX computers in telephones doesn't change what
"embedded" has always meant, or at least implied. The system I'm
working on now is built around an ARM7 at 70MHz with 512KB of flash and
32KB of RAM, and that's about as big as I go in the embedded space. It
doesn't have file-structured mass storage, nor does it have a user
interface in any traditional sense. THAT'S embedded.
-Dave
--
Dave McGuire
New Kensington, PA
I have no idea what embedded systems are like these
days , they used
to be processor, storage, and RAM starved so they did need optimized
but you also knew exactly what the resources were.
They still generally are processor, storage and RAM starved these
days, but the spectrum is a lot larger on the top end.
10:44 p.m.
New subject: Performance-oriented development in the commercial computing world
On 26 Oct 2011 at 1:26, Dave McGuire wrote:
"embedded" has always meant, or at least
implied. The system I'm
working on now is built around an ARM7 at 70MHz with 512KB of flash
and 32KB of RAM, and that's about as big as I go in the embedded
space. It doesn't have file-structured mass storage, nor does it have
a user interface in any traditional sense. THAT'S embedded.
So why the blurb about the ARM A15 in "embedded" applications in
today's email (from ED "embedded update")?
Methinks the goalposts are being moved...
--Chuck
10:56 p.m.
New subject: Performance-oriented development in the commercial computing world
On 10/26/2011 01:44 AM, Chuck Guzis wrote:
Lots of newbies (even newbies who have chops...just meaning people
who haven't been doing this for awhile) get the idea that "embedded"
involves hundreds of MHz of clock speed and megabytes of memory.
They haven't moved MY goal posts.
But then, this is illustrative of the fact that there are several
"embedded worlds" that are separate and distinct from one another.
-Dave
--
Dave McGuire
New Kensington, PA
"embedded" has always meant, or at least implied. The system I'm
working on now is built around an ARM7 at 70MHz with 512KB of flash
and 32KB of RAM, and that's about as big as I go in the embedded
space. It doesn't have file-structured mass storage, nor does it have
a user interface in any traditional sense. THAT'S embedded.
So why the blurb about the ARM A15 in "embedded" applications in
today's email (from ED "embedded update")?
Methinks the goalposts are being moved...
11:13 p.m.
New subject: Performance-oriented development in the commercial computing world
On 10/26/2011 01:56 AM, Dave McGuire wrote:
But then, this is illustrative of the fact that there
are several
"embedded worlds" that are separate and distinct from one another.
I probably should've elaborated on this a bit.
I see three distinct "embedded worlds".
The first involves tiny processors, usually 8-bitters, in 8- to
16-pin packages, clocked at a few kHz to a few MHz at most, with a few
dozen bytes of RAM. These embedded systems are found in your TV remote
controls, power managers for laptops, electric utility meters, air
conditioners, etc. They are typically programmed to the bare metal.
The second involves bigger processors, typically 8, 16, or even 32
bits, clocked below 100MHz with less than 1MB of RAM. They sometimes
use an RTOS, but they don't run UNIX. (this is where I work the most)
The third involves BIG processors, sometimes even x86, usually
running Linux or an embedded BSD variant, but almost always some
full-blown UNIX-flavored OS.
The stratification between these three categories seems to be pretty
clear these days, and some people "cross over" from time to time as
required.
-Dave
--
Dave McGuire
New Kensington, PA
26 Oct
26 Oct
8:05 a.m.
New subject: Performance-oriented development in the commercial computing world
On Oct 26, 2011, at 1:26 AM, Dave McGuire wrote:
On 10/26/2011 12:45 AM, Chuck Guzis wrote:
I think it's a highly subjective term you'll find a lot of people arguing about.
My gut-level definition is any computer that's designed not to seem like a computer,
which covers any number of industrial control applications, consumer devices that
aren't computers (like the army of talking baby toys that are out now), etc. I'd
personally make the distinction between "embedded" and "mobile" when
there's a screen attached.
Of course, where I find the lines blurry are things like the application I mentioned,
which was basically a bunch of processing blades. The board (an AMC board, if anyone is
familiar) was just a 2"x6" (roughly) board with a big-ass FPGA, a TI DSP (why
they insisted on that is anyone's guess) and the PowerPC for housekeeping. It was
primarily designated for processing satellite data, with the FPGA doing the heavy DSP
lifting and the DSP doing the cellular signal processing.
So what do we call that? Maybe it's not embedded, but it's certainly highly
specialized hardware and software for a particular task (you could use that board for any
number of heavy compute tasks, but it would make a terrible PC or generic server).
Just musing, anyway. I feel like going down this taxonomy rabbit hole is as much of a
fool's errand as the "speed metal"/"thrash metal"/etc. arguments
we'd have in high school.
- Dave
The problem is that I'm not sure what
"embedded" means any more. Is
a smartphone with a Cortex A15 4-core processor "embedded"? How
about an Android Pad?
The distinction is blurring pretty fast.
Putting full-blown UNIX computers in telephones doesn't change what
"embedded" has always meant, or at least implied. The system I'm working on
now is built around an ARM7 at 70MHz with 512KB of flash and 32KB of RAM, and that's
about as big as I go in the embedded space. It doesn't have file-structured mass
storage, nor does it have a user interface in any traditional sense. THAT'S embedded.
9:20 a.m.
New subject: Performance-oriented development in the commercial computing world
On 26 Oct 2011 at 11:05, David Riley wrote:
Just musing, anyway. I feel like going down this
taxonomy rabbit hole
is as much of a fool's errand as the "speed metal"/"thrash
metal"/etc.
arguments we'd have in high school.
Exactly. We're a long way from the time that "embedded" meant 68HC11
or 8051-style applications. I know that there's at least one vendor
that sticks a Linux system inside what amounts to a 100BaseT female
connector. Is it embedded? I'd say so.
The absence or presence of file systems is neither here nor there. I
wouldn't dispute that a $20 MP3 player, for example, is an embedded
application, but there's a file system in there for certain.
Operating system? Well, there are OSes that fit inside a low-end
PIC. For that matter, there's a version of Linux that runs on a
PIC32 ?C.
Rabbit-hole indeed. The boundaries are very indistinct.
Particularly when someone is talking about internet-connected home
refrigerators.
Technology has a tendency to obsolete terms of art. On my local
Craigslist, there's a seller who advertises a "minicomputer" for
sale. It's basically an mobile device with an external keyboard.
--Chuck
9:41 a.m.
New subject: Performance-oriented development in the commercial computing world
On Oct 26, 2011, at 12:20 PM, Chuck Guzis wrote:
Operating system? Well, there are OSes that fit
inside a low-end
PIC. For that matter, there's a version of Linux that runs on a
PIC32 ?C.
To be fair, a "PIC32" is just what Microchip has decided to call its MIPS
microcontroller line. It's as much a PIC as an SGI box is (aside from the form factor
and reduced RAM; most PIC32s have as many Ethernet interfaces and many times more serial
ports than most SGI machines ever did, even if they require external PHYs to talk to the
outside world).
Not relevant to the discussion, but it hurts my brain a little when I see people talk
about PIC32s as somehow related to PICs. They're great micros, but I wish Microchip
had called them something else. :-)
- Dave
10:26 a.m.
New subject: Performance-oriented development in the commercial computing world
On 10/26/2011 12:41 PM, David Riley wrote:
Agreed 100%.
I really wish the relevant GCC port maintainers would get their stuff
straightened out for that platform.
-Dave
--
Dave McGuire
New Kensington, PA
Operating
system? Well, there are OSes that fit inside a low-end
PIC. For that matter, there's a version of Linux that runs on a
PIC32 ?C.
To be fair, a "PIC32" is just what Microchip has decided to call its MIPS
microcontroller line. It's as much a PIC as an SGI box is (aside from the form factor
and reduced RAM; most PIC32s have as many Ethernet interfaces and many times more serial
ports than most SGI machines ever did, even if they require external PHYs to talk to the
outside world).
Not relevant to the discussion, but it hurts my brain a little when I see people talk
about PIC32s as somehow related to PICs. They're great micros, but I wish Microchip
had called them something else. :-)
1:48 p.m.
New subject: Performance-oriented development in the commercial computing world
On 26 Oct 2011 at 12:41, David Riley wrote:
On Oct 26, 2011, at 12:20 PM, Chuck Guzis wrote:
No argument there; a substantial chunk of the PIC32 documentation
appears to be cut and paste from the R4000 docs. But also to be
fair, Digilent markets a PIC32-based replacement for the AVR-based
Arduino, which is pretty much the embodiment of an embedded ?C.
In other words, the CPU architecture doesn't define the embeddedness.
--Chuck
Operating system? Well, there are OSes that fit
inside a low-end
PIC. For that matter, there's a version of Linux that runs on a
PIC32 ?C.
To be fair, a "PIC32" is just what Microchip has decided to call its
MIPS microcontroller line. It's as much a PIC as an SGI box is (aside
from the form factor and reduced RAM; most PIC32s have as many
Ethernet interfaces and many times more serial ports than most SGI
machines ever did, even if they require external PHYs to talk to the
outside world).
8:17 p.m.
New subject: Performance-oriented development in the commercial computing world
On Oct 26, 2011, at 4:48 PM, Chuck Guzis wrote:
No argument there; a substantial chunk of the PIC32
documentation
appears to be cut and paste from the R4000 docs. But also to be
fair, Digilent markets a PIC32-based replacement for the AVR-based
Arduino, which is pretty much the embodiment of an embedded ?C.
In other words, the CPU architecture doesn't define the embeddedness.
Absolutely agreed, I was just getting tetchy about the juxtaposition of the talk about
little RTOSes for PICs and then Unix on the PIC32. It's a nervous tic I have from
working with too many customers whose only experience with micros is the PIC (because
they're too lazy to try anything else) and insist on upgrading to a PIC32 because they
think it'll be compatible with their existing software base.
I know you're smarter than that, but seeing both items together just makes those
klaxons start going off in my head...
- Dave
10:25 a.m.
New subject: Performance-oriented development in the commercial computing world
On 10/26/2011 12:20 PM, Chuck Guzis wrote:
Yeah, I really can't disagree with that. That's only how I tend to
think of things. One really has to consider them on a case-by-case
basis. There's at least one popular MP3 player that will give you a
UNIX shell prompt if you frob it just right. (iPod Touch)
Kinda like the "what constitutes a minicomputer" debate. Nobody
disagrees that, say, PDP-11s are minis, but nobody can coherently tell
you why. People who know about this stuff just "know".
Just musing,
anyway. I feel like going down this taxonomy rabbit hole
is as much of a fool's errand as the "speed metal"/"thrash
metal"/etc.
arguments we'd have in high school.
Exactly. We're a long way from the time that "embedded" meant 68HC11
or 8051-style applications. I know that there's at least one vendor
that sticks a Linux system inside what amounts to a 100BaseT female
connector. Is it embedded? I'd say so.
The absence or presence of file systems is neither here nor there. I
wouldn't dispute that a $20 MP3 player, for example, is an embedded
application, but there's a file system in there for certain. Operating system? Well, there are OSes that fit
inside a low-end
PIC. For that matter, there's a version of Linux that runs on a
PIC32 ?C.
Indeed, 2.11BSD has been ported to PIC32!
Rabbit-hole indeed. The boundaries are very
indistinct.
Particularly when someone is talking about internet-connected home
refrigerators.
Agreed 100%.
Technology has a tendency to obsolete terms of art.
On my local
Craigslist, there's a seller who advertises a "minicomputer" for
sale. It's basically an mobile device with an external keyboard.
Yes, that drives me nuts, though I suppose it's unavoidable. Is it
Maxim or DalSemi who sells ICs in a package called "Flip Chip"?
And no matter what terms of art become obsolete, we are doomed to
argue about them here. ;)
-Dave
--
Dave McGuire
New Kensington, PA
10:55 a.m.
New subject: Performance-oriented development in the commercial computing world
On Oct 26, 2011, at 1:25 PM, Dave McGuire wrote:
Yes, that drives me nuts, though I suppose it's
unavoidable. Is it Maxim or DalSemi who sells ICs in a package called "Flip
Chip"?
"Flip chip" is actually a pretty accepted term for IC packaging these days
(referring to ICs which attach directly to their package substrate via bumps on the bottom
rather than wirebonds). I can see why it'd drive you crazy, though... DEC had theirs
before ICs were even a useful product!
- Dave
27 Oct
27 Oct
6:25 a.m.
New subject: Performance-oriented development in the commercial computing world
Dave McGuire wrote:
Yes, that drives me nuts, though I suppose it's
unavoidable. Is it Maxim
or DalSemi who sells ICs in a package called "Flip Chip"?
Also Intel with some Pentium-IIIs.
Peace... Sridhar
26 Oct
26 Oct
10:56 a.m.
New subject: Performance-oriented development in the commercial computing world
On 10/26/2011 11:05 AM, David Riley wrote:
I think it's a highly subjective term you'll find a lot of people
arguing about. My gut-level definition is any computer that's
designed not to seem like a computer, which covers any number of
industrial control applications, consumer devices that aren't
computers (like the army of talking baby toys that are out now), etc.
I'd personally make the distinction between "embedded" and
"mobile"
when there's a screen attached.
Of course, where I find the lines blurry are things like the
application I mentioned, which was basically a bunch of processing
blades. The board (an AMC board, if anyone is familiar) was just a
2"x6" (roughly) board with a big-ass FPGA, a TI DSP (why they
insisted on that is anyone's guess) and the PowerPC for housekeeping.
It was primarily designated for processing satellite data, with the
FPGA doing the heavy DSP lifting and the DSP doing the cellular
signal processing.
So what do we call that? Maybe it's not embedded, but it's certainly
highly specialized hardware and software for a particular task (you
could use that board for any number of heavy compute tasks, but it
would make a terrible PC or generic server).
Yes, this is sorta the point I was trying to make in my earlier email
on this thread. We just sorta "know" whether something is "embedded"
(or a "minicomputer") or not; the lines aren't quite so sharp these days
as Chuck observed.
The problem is that I'm not sure what
"embedded" means any more.
Is a smartphone with a Cortex A15 4-core processor "embedded"?
How about an Android Pad?
The distinction is blurring pretty fast.
Putting full-blown UNIX computers in telephones doesn't change what
"embedded" has always meant, or at least implied. The system I'm
working on now is built around an ARM7 at 70MHz with 512KB of flash
and 32KB of RAM, and that's about as big as I go in the embedded
space. It doesn't have file-structured mass storage, nor does it
have a user interface in any traditional sense. THAT'S embedded. Just musing, anyway. I feel like going down this
taxonomy rabbit
hole is as much of a fool's errand as the "speed metal"/"thrash
metal"/etc. arguments we'd have in high school.
Yeah, it's a mess. We're famous for that particular type of mess on
this list. ;) People who have never stood in the same room with a
minicomputer before have argued about what constitutes one. Fun!
-Dave
--
Dave McGuire
New Kensington, PA
10:27 a.m.
New subject: Performance-oriented development in the commercial computing world
In article <D7B8C9701BEB421D83F6EAB5FE505EA0 at dell8300>,
"TeoZ" <teoz at neo.rr.com> writes:
I would guess it depends on what you are doing.
No shit.
Console developers know that their target platform is
not moving at all, so
if they expect the next games to be any better or faster they HAVE to
optimize or wait for the next console (and lose sales).
Game development is limited by content creation these days, *NOT*
engine optimization.
I have no idea what embedded systems are like these
days, [...]
I do and the idea that sticking your developers on a resource limited
machine will make them produce better code for the embedded
environment is stupid and noone does it that way.
If you are programming shareware utilities you might
want the code to work
on a large range of machines and operating system versions so code size and
speed might be optimized to work on older slower gear but you also might not
be able to user tools that need specific OS revisions to work.
Code size and speed are largely irrelevant to keeping the OS
compatability matrix working.
--
"The Direct3D Graphics Pipeline" -- DirectX 9 draft available for download
<http://legalizeadulthood.wordpress.com/the-direct3d-graphics-pipeline/>
Legalize Adulthood! <http://legalizeadulthood.wordpress.com>
11:04 a.m.
New subject: Performance-oriented development in the commercial computing world
On 10/26/2011 01:27 PM, Richard wrote:
You're right that noone does it that way, but it's not because it's
"stupid" (which it isn't), it's because developers don't WANT to
work on
slow machines.
I sure don't. Do you?
Hardly. Every new function is a potential compatibility nit.
Richard, your attitude in the past few days has been even worse than
mine, and that's saying something. Is everything ok?
-Dave
--
Dave McGuire
New Kensington, PA
I have no idea
what embedded systems are like these days, [...]
I do and the idea that sticking your developers on a resource limited
machine will make them produce better code for the embedded
environment is stupid and noone does it that way. If you are
programming shareware utilities you might want the code to work
on a large range of machines and operating system versions so code size and
speed might be optimized to work on older slower gear but you also might not
be able to user tools that need specific OS revisions to work.
Code size and speed are largely irrelevant to keeping the OS
compatability matrix working.
29 Oct
29 Oct
9:30 p.m.
New subject: Performance-oriented development in the commercial computing world
[I]t's not because it's "stupid"
(which it isn't), it's because
developers don't WANT to work on slow machines.
I sure don't. Do you?
Other things being equal, I don't.
Other things are never equal. There are four reasons that come to
mind immediately why I'd rather work on my (comparatively) slow SS20
rather than some ridiculously overmuscled "modern" machine.
Specifically, (1) so I can use a good keyboard (I've managed to hack a
peecee X server to take a Sun keyboard on a serial port; it mostly
works but has some annoying issues); (2) I have an aesthetic dislike
for the x86 architecture, and in particular really do not appreciate
wasing a substantial fraction of the silicon (and power) on an
emulation layer that just keeps me away from the full power of the
hardware; (3) working on slow machines keeps me honest, preventing me
from sweeping my coding sins under the rug with
hardware oomph; (4) the
SS20 is substantially more reliable hardware.
Obviously, these won't all apply to everyone. Some people, _none_ of
them will apply to. But those are (some of) my reasons.
/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTML mouse at rodents-montreal.org
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
9:50 p.m.
New subject: Performance-oriented development in the commercial computing world
On 10/30/2011 12:30 AM, Mouse wrote:
Other things are never equal. There are four reasons
that come to
mind immediately why I'd rather work on my (comparatively) slow SS20
rather than some ridiculously overmuscled "modern" machine.
Specifically, (1) so I can use a good keyboard (I've managed to hack a
peecee X server to take a Sun keyboard on a serial port; it mostly
works but has some annoying issues); (2) I have an aesthetic dislike
for the x86 architecture, and in particular really do not appreciate
wasing a substantial fraction of the silicon (and power) on an
emulation layer that just keeps me away from the full power of the
hardware; (3) working on slow machines keeps me honest, preventing me
from sweeping my coding sins under the rug with hardware oomph; (4) the
SS20 is substantially more reliable hardware.
Obviously, these won't all apply to everyone. Some people, _none_ of
them will apply to. But those are (some of) my reasons.
I wish more developers were like you. Our day-to-day computing
experience would be so much nicer if they were.
-Dave
--
Dave McGuire
New Kensington, PA
10:37 p.m.
New subject: Performance-oriented development in the commercial computing world
> Other things are never equal. There are four
reasons that come to
> mind immediately [...]
> Obviously, these won't all apply to everyone.
Some people, _none_
> of them will apply to. But those are (some of) my reasons.
I wish more developers were like you. Our day-to-day
computing
experience would be so much nicer if they were.
Thank you. That's the nicest thing anyone has said to me (on a mailing
list, at least) in...quite a while.
/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTML mouse at rodents-montreal.org
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
30 Oct
30 Oct
7:27 a.m.
New subject: Performance-oriented development in the commercial computing world
On 30/10/11 12:30 AM, Mouse wrote:
Exactly.
--Toby
> [I]t's not because it's "stupid"
(which it isn't), it's because
> developers don't WANT to work on slow machines.
Wouldn't make much difference to me. Because, at least when I do it, my
activities are:
1) A *lot* of thinking
2) A little bit of typing
and
3) Stuff-all else that taxes a machine (the most demanding thing would
be a web browser and occasional unit test runs).
Are there common things that a programmer waits for? Compiles have
hardly been a burden in the past decade... The machine should nearly
always be waiting for the programmer while they think about what they
are doing. ("More thought, less code" is a whole other rant.)
I'd be equally productive on a 10 year old box; but I'd have the benefit
of knowing if I were doing something silly, performance wise. A poorly
phrased database query in a web app will be a problem on the fastest
server, as client load increases. And judging by the bloatware around,
examples surely abound in client side development. How can fast testing
machines not hide performance problems? Performance/resource bugs are
bugs too.
The exception seems to be the arms race in JavaScript engine
performance. If we could set up this kind of competition in many more
areas, we might see software that doesn't demand a 6 month old PC (not
forgetting all those long-standing negative feedback loops that ask you
to throw out perfectly good hardware, buy new kit you don't need, along
with one more trip through the Windows tollbooth).
I sure
don't. Do you?
Other things being equal, I don't.
Other things are never equal. There are four reasons that come to
mind immediately why I'd rather work on my (comparatively) slow SS20
rather than some ridiculously overmuscled "modern" machine.
Specifically, (1) so I can use a good keyboard (I've managed to hack a
peecee X server to take a Sun keyboard on a serial port; it mostly
works but has some annoying issues); (2) I have an aesthetic dislike
for the x86 architecture, and in particular really do not appreciate
wasing a substantial fraction of the silicon (and power) on an
emulation layer that just keeps me away from the full power of the
hardware; (3) working on slow machines keeps me honest, preventing me
from sweeping my coding sins under the rug with hardware oomph; (4) the
SS20 is substantially more reliable hardware.
Obviously, these won't all apply to everyone. Some people, _none_ of
them will apply to. But those are (some of) my reasons.
/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTML mouse at rodents-montreal.org
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
9:09 a.m.
New subject: Performance-oriented development in the commercial computing world
Toby Thain [toby at telegraphics.com.au]:
Are there common things that a programmer waits for?
Compiling a few directories takes seconds, but linking takes
a little longer. Perhaps 30s on my 24GB system at work but
more like 2-3m in its previous 8GB incarnation.
There must be plenty of projects where a top-level build takes
quite a while.
In our case giving developers slow machines wouldn't help the
final product since it runs on specific hardware. That must
be true of a fair chunk of embedded development today (network
boxes, phones, consoles ...).
I agree that *testing* an end-user application on the least capable
acceptable hardware is a good idea. Even there though, I don't see
how forcing the developer to use that hardware for development
achieves anything other than slowing them down.
Antonio
arcarlini at iee.org
10:19 a.m.
New subject: Performance-oriented development in the commercial computing world
Toby Thain [toby at telegraphics.com.au]:
Are there common things that a programmer waits for?
Reminds me of a discussion that I had with Greg Mansfield, then of
Cray sometime around 1983. We were discussing mods to Unix (I, BSD
for the VAX; he, UniCOS) and I happened to mention that maintiang the
makefiles and integrity of partial builds was a bit of a burden. He
responded that he didn't bother with partial builds--he just
recompiled the whole thing every time.
After all, he was running on some very fast hardware. Indeed, even
today, I'll find myself wondering what the heck I'm doing working out
a makefile for a project whose compilation time is a minute at worst.
And I agree--time spent away from the damned box is probably more
important than time spent in front of it in the development process.
For me, time spent sleeping seems to be my most productive time I
find that designing software when fatigued results in many mistakes
and foggy thought.
My most valuable development tool remains a quadrule pad and a
pencil.
My most valuable lesson learned is *never* give in to management
pressure to cut development time on a project. We all could take
lessons from Michaelangelo there.
--Chuck
11:05 a.m.
New subject: Performance-oriented development in the commercial computing world
On 30/10/11 1:19 PM, Chuck Guzis wrote:
Toby Thain [toby at telegraphics.com.au]:
One increasingly important reason: parallel builds. A correct makefile
gives you that for free.
Even to correctly rebuild "the world" in parallel (and if you're doing
that, you WANT it happening on multiple cores) will require a correct
makefile. It's a worthwhile investment. Plus you never know when you (or
some random end user, in the open source universe) will want to build on
a verrrrry slow machine. They'll curse you if you only ever built on an
8 core Xeon and never gave a damn about incremental builds or
efficiency. (Another justification of the original proposal to keep dev
boxes slow: or you might never end up with sensible makefiles.)
Are there common things that a programmer waits for?
Reminds me of a discussion that I had with Greg Mansfield, then of
Cray sometime around 1983. We were discussing mods to Unix (I, BSD
for the VAX; he, UniCOS) and I happened to mention that maintiang the
makefiles and integrity of partial builds was a bit of a burden. He
responded that he didn't bother with partial builds--he just
recompiled the whole thing every time.
After all, he was running on some very fast hardware. Indeed, even
today, I'll find myself wondering what the heck I'm doing working out
a makefile for a project whose compilation time is a minute at worst.
And I agree--time spent away from the damned box is probably more
important than time spent in front of it in the development process.
Thanks for concurring; I was wondering if I was the only person on the
list who felt thinking, design, and reasoning were important parts of
programming.
For me, time spent sleeping seems to be my most
productive time I
find that designing software when fatigued results in many mistakes
and foggy thought.
Oh yes, can't count the times when I've burned out on an issue at 6pm
and solved it by 10am with a fresh mind the next morning.
My most valuable development tool remains a quadrule pad and a
pencil.
Mine too!
My most valuable lesson learned is *never* give in to management
pressure to cut development time on a project. We all could take
lessons from Michaelangelo there.
The funny thing is, management never learns. They get burned by cutting
a time budget, fail to join the dots, and do it over and over again.
Lessons from Michaelangelo and the late great Fred Brooks, indeed.
--Toby
--Chuck
12:08 p.m.
New subject: Performance-oriented development in the commercial computing world
On 10/30/2011 02:05 PM, Toby Thain wrote:
YES! That happened to me *just yesterday*. The annoying thing is
that I often can't tell that I'm actually that tired.
-Dave
--
Dave McGuire
New Kensington, PA
For me, time
spent sleeping seems to be my most productive time I
find that designing software when fatigued results in many mistakes
and foggy thought.
Oh yes, can't count the times when I've burned out on an issue at 6pm
and solved it by 10am with a fresh mind the next morning.
12:15 p.m.
New subject: Performance-oriented development in the commercial computing world
On 30 Oct 2011 at 14:05, Toby Thain wrote:
One increasingly important reason: parallel builds. A
correct makefile
gives you that for free.
We'd do it at the source level years ago, using a source library
program such as UPDATE. It's often the case that so many
structures and procedures are shared by routines in the OS, that a
full recompile was about as effective as a partial build.
An extreme case of compiles taking a long time was when I was doing
STAR OS development and the only (convenient) resource was a pair of
STAR-1B systems (the only extant ones), that ran at somewhere around
1/1000th the speed of the real thing. They were not intended as
standard line products, but rather as development platforms that had
the same core (256K words), and attached to standard STAR SBUs and
MCU, but were otherwise bit-serial heavily microprogrammed
emulators..
A compile of the OS kernel was an overnight job, taking a bit more
than 8 hours. That's the kernel, not any of the system utilities.
More often than not, the hardware would develop problems and you'd
lose a day.
You quickly learned that desk-checking everything thoroughly would
save huge amounts of time. It was a very frustrating way to work.
There were only two other options--beg some maintenance time from the
CEs at Livermore, or grab a flight to Minneapolis/St. Paul to use the
system at ADL in Arden Hills to recompile.
This was quite possibly the worst development environment that I'd
ever worked in.
--Chuck
12:03 p.m.
New subject: Performance-oriented development in the commercial computing world
On 10/30/2011 01:19 PM, Chuck Guzis wrote:
Reminds me of a discussion that I had with Greg
Mansfield, then of
Cray sometime around 1983. We were discussing mods to Unix (I, BSD
for the VAX; he, UniCOS) and I happened to mention that maintiang the
makefiles and integrity of partial builds was a bit of a burden. He
responded that he didn't bother with partial builds--he just
recompiled the whole thing every time.
After all, he was running on some very fast hardware. Indeed, even
today, I'll find myself wondering what the heck I'm doing working out
a makefile for a project whose compilation time is a minute at worst.
That sorta makes sense I suppose. I was brought up with a "waste
not, want not" mentality, and that extends to my development process. I
have a few "boilerplate" Makefiles that I keep around for starting new
projects; it takes me about a minute to set one up and get it running.
Maybe five minutes for a complicated one like for cross-building ARM7 stuff.
-Dave
--
Dave McGuire
New Kensington, PA
10:26 a.m.
New subject: Performance-oriented development in the commercial computing world
On 30/10/11 12:09 PM, arcarlini at iee.org wrote:
Toby Thain [toby at telegraphics.com.au]:
Sometimes due to incompetent build systems (in my experience). I am sure
you've seen this Makefile more than once:
exe : <about two hundred .c files>
$(CC) -o $@ $^
Are there common things that a programmer waits
for?
Compiling a few directories takes seconds, but linking takes
a little longer. Perhaps 30s on my 24GB system at work but
more like 2-3m in its previous 8GB incarnation.
There must be plenty of projects where a top-level build takes
quite a while.
In our case giving developers slow machines wouldn't help the
final product since it runs on specific hardware. That must
be true of a fair chunk of embedded development today (network
boxes, phones, consoles ...).
That's a thoroughly valid exception, indeed.
I agree that *testing* an end-user application on the least capable
acceptable hardware is a good idea. Even there though, I don't see
how forcing the developer to use that hardware for development
achieves anything other than slowing them down.
Then how would *you* solve the perennial problem of software whose
pointless sloth and bloat won't allow it to run well on anything but the
latest hardware?
It is hard to dispute that a) older hardware gives the developer better
visibility to performance issues (it changes their priorities, even
subconsciously), and b) many developers do not need the "latest"
hardware at all. Why would it slow their *thinking* down? They should be
doing more thinking than typing, and a slow machine doesn't slow typing
down either...
The thing is: Make something run acceptably on the bulk of machines out
there (which are not cutting edge) and you improve things for those who
want to run the latest gadgets, commensurately. Looks like a win-win to me.
--Toby
Antonio
arcarlini at iee.org
3:39 p.m.
New subject: Performance-oriented development in the commercial computing world
Toby Thain [toby at telegraphics.com.au] wrote:
My solution is in the bit you conveniently quoted above :-)
Despite being the consensus here, it doesn't appear to work that well.
If this were a problem for the majority of customers, then the market
would solve the issue
(i.e. someone else would release a competing product that didn't have
this issue but otherwise
worked at least "well enough").
So either the majority of customers do not see this problem or do not
see it as a problem or
the market is in some way broken. I'm not sure which of these is
responsible, or even if it
is a combination of these factors. I do know that if a business finds
that application $FOO
just won't run on whatever desktops they have, then almost certainly it
seems to be cheaper
to replace those desktops with a new ?500 dual-core desktop. (I say that
even though the
environment I work in develops using Linux-based systems and we're happy
to "roll our own"
whenever it is required - most places just don't work that way probably
because most places
are not development shops).
Antonio
arcarlini at iee.org
On 30/10/11 12:09 PM, arcarlini at iee.org wrote:
Ours is just big.
There must be plenty of projects where a
top-level build
takes quite a
while.
Sometimes due to incompetent build systems (in my
experience). I am sure
you've seen this Makefile more than once:
exe : <about two hundred .c files>
$(CC) -o $@ $^ That's a thoroughly valid exception, indeed.
The embedded/dedicated hardware stuff isn't an exception - it's most of
my career :-)
I agree that *testing* an end-user application on the least capable
acceptable hardware is a good idea. Even there though, I
don't see how
forcing the developer to use that hardware for
development achieves
anything other than slowing them down.
Then how would *you* solve the perennial problem of software whose
pointless sloth and bloat won't allow it to run well on
anything but the
latest hardware?
5:05 p.m.
New subject: Performance-oriented development in the commercial computing world
On 30/10/11 6:39 PM, arcarlini at iee.org wrote:
Toby Thain [toby at telegraphics.com.au] wrote:
True, but as you go on to say, it hasn't worked.
On 30/10/11 12:09 PM, arcarlini at iee.org
wrote:
My solution is in the bit you conveniently quoted above :-) ...
I agree that *testing* an end-user application on the least capable
acceptable hardware is a good idea. Even there though, I
don't see how
forcing the developer to use that hardware for
development achieves
anything other than slowing them down.
Then how would *you* solve the perennial problem of software whose
pointless sloth and bloat won't allow it to run well on
anything but the
latest hardware?
Despite being the consensus here, it doesn't appear to work that well.
Maybe because nobody does it. The norm is forced upgrades?
--T
If this were a problem for the majority of customers,
then the market
would solve the issue
(i.e. someone else would release a competing product that didn't have
this issue but otherwise
worked at least "well enough").
...
Antonio
arcarlini at iee.org
5:22 p.m.
New subject: Performance-oriented development in the commercial computing world
On 30/10/11 6:39 PM, arcarlini at iee.org wrote:
Toby Thain [toby at telegraphics.com.au] wrote:
But of course it isn't even an exception. It fits the rule perfectly:
'Test on what people actually use, not on what was purchased arbitrarily
for whiny developers.'
If nobody used anything less than a 2011 PC, the problem wouldn't exist. :)
--T
On 30/10/11 12:09 PM, arcarlini at iee.org
wrote:
Ours is just big.
There must be plenty of projects where a
top-level build
takes quite a
while.
Sometimes due to incompetent build systems (in my
experience). I am sure
you've seen this Makefile more than once:
exe :<about two hundred .c files>
$(CC) -o $@ $^ That's a thoroughly valid exception, indeed.
The embedded/dedicated hardware stuff isn't an exception - it's most of
my career :-)
5:58 p.m.
New subject: Performance-oriented development in the commercial computing world
On 30 Oct 2011 at 20:22, Toby Thain wrote:
But of course it isn't even an exception. It fits
the rule perfectly:
'Test on what people actually use, not on what was purchased
arbitrarily for whiny developers.'
This brings up an interesting question. Has compilation speed
generally kept pace with Moore's law? Getting C compilation speeds
around 5000 lpm wasn't that unusual using a 5MHz 8088 with a hard
disk. Borland Delphi running on a P90 claimed 350,000 lines per
minute.
Could one infer that today's compilation speeds are in the range of
tens to hundreds of millions of lines per minute? Do any vendors
even advertise a number?
--Chuck
6:57 p.m.
New subject: Performance-oriented development in the commercial computing world
On 10/30/11 5:58 PM, "Chuck Guzis" <cclist at sydex.com> wrote:
On 30 Oct 2011 at 20:22, Toby Thain wrote:
I know that on sparc Niagra architecture (Sunacle T2000, T5120), GCC does
not come even close to tens or hundeds of millions of lines per minute :(
Heck I don't think it even gets close to a million lines in a minute.
But of course it isn't even an exception. It
fits the rule perfectly:
'Test on what people actually use, not on what was purchased
arbitrarily for whiny developers.'
This brings up an interesting question. Has compilation speed
generally kept pace with Moore's law? Getting C compilation speeds
around 5000 lpm wasn't that unusual using a 5MHz 8088 with a hard
disk. Borland Delphi running on a P90 claimed 350,000 lines per
minute.
Could one infer that today's compilation speeds are in the range of
tens to hundreds of millions of lines per minute? Do any vendors
even advertise a number?
--Chuck
7:35 p.m.
New subject: how fast is gcc? - Re: Performance-oriented development in the commercial computing world
On 30/10/11 9:57 PM, Geoffrey Reed wrote:
On 10/30/11 5:58 PM, "Chuck Guzis"<cclist
at sydex.com> wrote:
What if you ran it on more than one core? Just a thought... :D
--T
On 30 Oct 2011 at 20:22, Toby Thain wrote:
I know that on sparc Niagra architecture (Sunacle T2000, T5120), GCC does
not come even close to tens or hundeds of millions of lines per minute :(
Heck I don't think it even gets close to a million lines in a minute. But of course it isn't even an exception. It
fits the rule perfectly:
'Test on what people actually use, not on what was purchased
arbitrarily for whiny developers.'
This brings up an interesting question. Has compilation speed
generally kept pace with Moore's law? Getting C compilation speeds
around 5000 lpm wasn't that unusual using a 5MHz 8088 with a hard
disk. Borland Delphi running on a P90 claimed 350,000 lines per
minute.
Could one infer that today's compilation speeds are in the range of
tens to hundreds of millions of lines per minute? Do any vendors
even advertise a number?
--Chuck
31 Oct
31 Oct
6:27 a.m.
New subject: how fast is gcc? - Re: Performance-oriented development in the commercial computing world
On Oct 30, 2011, at 10:35 PM, Toby Thain wrote:
>> This brings up an interesting question. Has
compilation speed
>> generally kept pace with Moore's law? Getting C compilation speeds
>> around 5000 lpm wasn't that unusual using a 5MHz 8088 with a hard
>> disk. Borland Delphi running on a P90 claimed 350,000 lines per
>> minute.
>>
>> Could one infer that today's compilation speeds are in the range of
>> tens to hundreds of millions of lines per minute? Do any vendors
>> even advertise a number?
I'll say with a fair degree of confidence that compilers, like everything else, tend
to bloat with more processing power available. They also tend to slow down with the bloat
of the operating system (if disk accesses take more cycles for small operations, for
example). It's frustrating to those of us trying to run modern OSes on older
hardware; NetBSD on my Power Mac 9500 used to run like lightning, and now it's
painfully slow due to a combination of much slower disk accesses and increased overhead in
GCC 4.5 vs. the 3.x series. I don't even want to think how it'll run on my 68k
boxes.
Even on modern (but slower) hardware, it's a problem. Running compiles from the
ramdisk on OpenBSD on my 1.6GHz Atom runs about as fast per file (which really tends to be
the grain I measure against, since the setup/teardown of GCC can be the limiting factor on
slower machines) as my 500 MHz K6-2 did from hard disk ten years ago. Of course, that was
running Linux, so the comparison is not scientifically valid since we're comparing
blowfish to penguins. :-)
- Dave
7:21 a.m.
New subject: how fast is gcc? - Re: Performance-oriented development in the commercial computing world
I don't even want to think how it'll run on my
68k boxes.
I have been unhappy with the amount of bloat creeping into recent NetBSD
versions. For my internal systems where security is less of a concern, I
keep them on 1.x. My IIci running 1.5.2 has uptime measured in years.
--
------------------------------------ personal: http://www.cameronkaiser.com/ --
Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckaiser at floodgap.com
-- Do me a favour: don't breed. -- "Sledge Hammer!"
---------------------------
8:14 a.m.
New subject: how fast is gcc? - Re: Performance-oriented development in the commercial computing world
On 10/31/2011 10:21 AM, Cameron Kaiser wrote:
Same here. I've often thought about forking NetBSD around 1.6 or so,
but that'd be a huge amount of work, and who has the time..
The bloat has even gotten so bad that one of the early NetBSD core
developers, who was gainfully employed generating embedded versions of
NetBSD for consumer digital cameras and stuff for many years, dumped
NetBSD for Linux for those types of applications. Scary. (that's
second-hand info, but reliable)
-Dave
--
Dave McGuire
New Kensington, PA
I don't
even want to think how it'll run on my 68k boxes.
I have been unhappy with the amount of bloat creeping into recent NetBSD
versions. For my internal systems where security is less of a concern, I
keep them on 1.x. My IIci running 1.5.2 has uptime measured in years.
30 Oct
30 Oct
7:10 p.m.
New subject: Performance-oriented development in the commercial computing world
On 30/10/11 8:58 PM, Chuck Guzis wrote:
On 30 Oct 2011 at 20:22, Toby Thain wrote:
Has the concept of translating the source file blobs of your codebase
into blobs of object code then linking them all together into a
monolithic executable -- *every time you change the program* -- kept
pace with modern concepts of software development? I mean, that's a bit
primitive even relative to the state of the art in 1985*...
But of course it isn't even an exception. It
fits the rule perfectly:
'Test on what people actually use, not on what was purchased
arbitrarily for whiny developers.'
This brings up an interesting question. Has compilation speed
generally kept pace with Moore's law? Getting C compilation speeds
around 5000 lpm wasn't that unusual using a 5MHz 8088 with a hard
disk. Borland Delphi running on a P90 claimed 350,000 lines per
minute.
Could one infer that today's compilation speeds are in the range of
tens to hundreds of millions of lines per minute? Do any vendors
even advertise a number?
Phoronix seems to obsessively benchmark compilers. I'm not really sure why.
--T
* Genera, http://en.wikipedia.org/wiki/Genera_%28operating_system%29
Smalltalk-80, etc.
--Chuck
7:31 p.m.
New subject: Performance-oriented development in the commercial computing world
On 30 Oct 2011 at 22:10, Toby Thain wrote:
Has the concept of translating the source file blobs
of your codebase
into blobs of object code then linking them all together into a
monolithic executable -- *every time you change the program* -- kept
pace with modern concepts of software development? I mean, that's a
bit primitive even relative to the state of the art in 1985*...
No, but it's usually a good idea to recompile the whole thing
occasionally someone may have screwed up the makefile or even mis-
dated a file (it happens), particularly when a regression is
observed. It would be nice to know if it was going to take seconds
or hours... If seconds, why not recompile the thing as a whole all
the time? I'd really start to get worried if the result of a
complete compilation differed from an incremental one.
I assume that most commercial software operations do a clean
recompile from scratch for new releases. So if you have a billion
lines of code in your codebase (that's 30 solid years of 24x7 coding
at the rate of one line per second), is it going to take 10 minutes
(coffee break) or 10 days (dead from dehydration)...
--Chuck
31 Oct
31 Oct
4:29 a.m.
New subject: Performance-oriented development in the commercial computing world
On 30/10/11 10:31 PM, Chuck Guzis wrote:
On 30 Oct 2011 at 22:10, Toby Thain wrote:
So would I. I'd probably try to fix the makefile. :D
Has the concept of translating the source file
blobs of your codebase
into blobs of object code then linking them all together into a
monolithic executable -- *every time you change the program* -- kept
pace with modern concepts of software development? I mean, that's a
bit primitive even relative to the state of the art in 1985*...
No, but it's usually a good idea to recompile the whole thing
occasionally someone may have screwed up the makefile or even mis-
dated a file (it happens), particularly when a regression is
observed. It would be nice to know if it was going to take seconds
or hours... If seconds, why not recompile the thing as a whole all
the time? I'd really start to get worried if the result of a
complete compilation differed from an incremental one.
I assume that most commercial software operations do a clean
recompile from scratch for new releases. So if you have a billion
lines of code in your codebase (that's 30 solid years of 24x7 coding
I don't think much commercial software weighs in at 1,000,000,000 lines
- yet. But you'd need an efficient build system if it did.
But this conversation is about what developers do every day. Release
builds would often not be built on the developer's machine.
--Toby
at the rate of one line per second), is it going to
take 10 minutes
(coffee break) or 10 days (dead from dehydration)...
--Chuck
6:08 a.m.
New subject: Performance-oriented development in the commercial computing world
I suspect that most compilers have moved from a "generate code as
quickly as possible" strategy to a "generate as quick of code as
possible" strategy today. Seems like a good move, IMHO.
Mike
3:39 p.m.
New subject: Performance-oriented development in the commercial computing world
Toby Thain [toby at telegraphics.com.au] wrote:
I don't think much commercial software weighs in
at
1,000,000,000 lines
- yet. But you'd need an efficient build system if it did.
100M (I just checked) but that's everything (comments, generated code,
blank lines).
I've no idea at what rate it's been growing so I've no idea when it will
reach one billion lines.
Antonio
arcarlini at iee.org
7:25 a.m.
New subject: Performance-oriented development in the commercial computing world
On Oct 30, 2011, at 10:10 PM, Toby Thain wrote:
Has the concept of translating the source file blobs
of your codebase into blobs of object code then linking them all together into a
monolithic executable -- *every time you change the program* -- kept pace with modern
concepts of software development? I mean, that's a bit primitive even relative to the
state of the art in 1985*...
It's interesting you should mention that; years ago, Apple's XCode developer tools
(basically a nice IDE wrapping around a GCC toolchain) had a "zerolink" option
for development that didn't bother to link and instead treated each object file as a
dynamic library. It certainly saved a lot of time linking large programs.
It was nice for development if you had a program that took forever to link, but I found it
to be a bit of a pain in the ass for debugging because link errors (mainly undefined
symbols due to forgetting to include a .c file, which the compiler didn't catch but
the linker would) happened mysteriously at runtime instead of at link time. I'm sure
that could be solved by adding a separate (fast) pass just to resolve all references
instead of doing a full link, but Apple pulled zerolink out a while back anyway
(presumably because machines got fast enough that no one cared about the long link time
anymore).
It's certainly a paradigm I wouldn't mind seeing go the way of the dodo, but for
everyday use stuff, it's certainly a tradeoff in speed between the developer and the
user; if everything has to be linked at runtime (see: Java, which admittedly also has the
JIT overhead and manifold problems of its own), startup times for even simple applications
increase by several orders of magnitude, which drives average users up the wall.
- Dave
30 Oct
30 Oct
8:41 p.m.
New subject: Performance-oriented development in the commercial computing world
Could one infer that today's compilation speeds
are in the range of
tens to hundreds of millions of lines per minute?
One could.
I suspect one would come to an incorrect conclusion by doing so,
though, unless the contemplated situation involves using an 8088-era
compiler. I just now tried compiling my tar (11301 lines, 304572
bytes) on my fastest CPU (one core of a 1.8GHz Core 2 Duo) and it
compiled and linked in 1.8 seconds. That's a little over a third of a
million lpm, for reasonably modern gcc on a reasonably modern machine.
Unfortunatley I don't have an 8088-era compiler at ready hand to try.
/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTML mouse at rodents-montreal.org
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
9:08 p.m.
New subject: Performance-oriented development in the commercial computing world
On 30 Oct 2011 at 23:41, Mouse wrote:
Unfortunatley I don't have an 8088-era compiler at
ready hand to try.
I do have a copy of Lattice C 2.0, which is more-or-less K&R.
Unfortunately, it doesn't take any advantage of more advanced
processor properties or instructions, so the comparison might be a
little off.
--Chuck
9:32 a.m.
New subject: Performance-oriented development in the commercial computing world
On Sun, 30 Oct 2011, Toby Thain wrote:
Wouldn't make much difference to me. Because, at
least when I do it, my
activities are:
1) A *lot* of thinking
2) A little bit of typing
and
3) Stuff-all else that taxes a machine (the most demanding thing would be a
web browser and occasional unit test runs).
Are there common things that a programmer waits for?
Grepping the source tree. If I substantially change the behavior of
function X, I need to be sure it won't break any of the (often hundreds
of) call sites. Enough RAM to fit the entire source tree in the cache is
very helpful for this, and that can mean several GB...
Also, in the past several places I've worked the developer machine has a
mini version of the entire production enviromnent on it, so that one can
actually run meaningful tests. For me that currently means an apache
instance with about a dozen virtual hosts, memcached, half a dozen mysqld
instances, several processes doing background jobs, and 4 or 5 postfix
instances.
Alexey
10:57 a.m.
New subject: Performance-oriented development in the commercial computing world
On 30/10/11 12:32 PM, Alexey Toptygin wrote:
On Sun, 30 Oct 2011, Toby Thain wrote:
Yes, that problem is solved by RAM, and smarter indexing :)
In fact, it's a *perfect* illustration of how a fast machine can conceal
a very inefficient design! Grepping a source tree for an identifier is
orders of magnitude more wasteful than simply using an inverted index.
Wouldn't make much difference to me. Because,
at least when I do it,
my activities are:
1) A *lot* of thinking
2) A little bit of typing
and
3) Stuff-all else that taxes a machine (the most demanding thing would
be a web browser and occasional unit test runs).
Are there common things that a programmer waits for?
Grepping the source tree. If I substantially change the behavior of
function X, I need to be sure it won't break any of the (often hundreds
of) call sites. Enough RAM to fit the entire source tree in the cache is
very helpful for this, and that can mean several GB... Also, in the past several places I've worked the
developer machine has a
mini version of the entire production enviromnent on it, so that one can
actually run meaningful tests. For me that currently means an apache
instance with about a dozen virtual hosts, memcached, half a dozen
mysqld instances, several processes doing background jobs, and 4 or 5
postfix instances.
Sure, some environments are simply too big for a 10 year old machine.
I'd suggest that this is the minority.
--T
Alexey
11:25 a.m.
New subject: Performance-oriented development in the commercial computing world
On Sun, 30 Oct 2011, Toby Thain wrote:
On 30/10/11 12:32 PM, Alexey Toptygin wrote:
Do you have any examples of software that does this sort of indexing
that's not built into a heavyweight IDE? I've never seen any.
Alexey
On Sun, 30 Oct 2011, Toby Thain wrote:
Yes, that problem is solved by RAM, and smarter indexing :)
In fact, it's a *perfect* illustration of how a fast machine can conceal a
very inefficient design! Grepping a source tree for an identifier is orders
of magnitude more wasteful than simply using an inverted index. Are there common things that a programmer waits
for?
Grepping the source tree. If I substantially change the behavior of
function X, I need to be sure it won't break any of the (often hundreds
of) call sites. Enough RAM to fit the entire source tree in the cache is
very helpful for this, and that can mean several GB...
12:12 p.m.
New subject: Performance-oriented development in the commercial computing world
>> Grepping the source tree.
> In fact, it's a *perfect* illustration of how a fast machine can
> conceal a very inefficient design! Grepping a source tree for an
> identifier is orders of magnitude more wasteful than simply using an
> inverted index.
Yes - but that's ignoring the cost of creating/maintaining the index in
the first place. If you grep the tree less often than you make changes
that would require updating the index, it may be cheaper to pay the
cost at grep time rather than at index rebuild/update time.
Do you have any examples of software that does this
sort of indexing
that's not built into a heavyweight IDE? I've never seen any.
ctags(1). It's not a full index (it's definitions-only, for example),
but I think it's close enough to count.
/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTML mouse at rodents-montreal.org
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
12:14 p.m.
New subject: Performance-oriented development in the commercial computing world
On 10/30/2011 02:25 PM, Alexey Toptygin wrote:
Yes, that problem is solved by RAM, and smarter indexing :)
In fact, it's a *perfect* illustration of how a fast machine can
conceal a very inefficient design! Grepping a source tree for an
identifier is orders of magnitude more wasteful than simply using an
inverted index.
Do you have any examples of software that does this sort of indexing
that's not built into a heavyweight IDE? I've never seen any.
Ctags comes to mind. It's pretty simplistic and has been around for
decades, but it works well.
-Dave
--
Dave McGuire
New Kensington, PA
Are there
common things that a programmer waits for?
Grepping the source tree. If I substantially change the behavior of
function X, I need to be sure it won't break any of the (often hundreds
of) call sites. Enough RAM to fit the entire source tree in the cache is
very helpful for this, and that can mean several GB...
4:25 p.m.
New subject: Performance-oriented development in the commercial computing world
On 30/10/11 2:25 PM, Alexey Toptygin wrote:
On Sun, 30 Oct 2011, Toby Thain wrote:
It would be a straightforward vim or emacs extension (and there are
plenty of libraries that implement this - such as Lucene/Solr).
If IDEs are "heavyweight" or have no indexing short of grep (Eclipse
seems to grep), then that is, as I say, a perfect illustration of
heedless, unreasonable bloat.
--Toby
On 30/10/11 12:32 PM, Alexey Toptygin wrote:
Do you have any examples of software that does this sort of indexing
that's not built into a heavyweight IDE? I've never seen any. On Sun, 30 Oct 2011, Toby Thain wrote:
Yes, that problem is solved by RAM, and smarter indexing :)
In fact, it's a *perfect* illustration of how a fast machine can
conceal a very inefficient design! Grepping a source tree for an
identifier is orders of magnitude more wasteful than simply using an
inverted index. Are there common things that a programmer waits
for?
Grepping the source tree. If I substantially change the behavior of
function X, I need to be sure it won't break any of the (often hundreds
of) call sites. Enough RAM to fit the entire source tree in the cache is
very helpful for this, and that can mean several GB...
Alexey
3:44 p.m.
New subject: Performance-oriented development in the commercial computing world
Toby Thain [toby at telegraphics.com.au] wrote:
Yes, that problem is solved by RAM, and smarter
indexing :)
24GB does help :-)
In fact, it's a *perfect* illustration of how a
fast machine
can conceal
a very inefficient design! Grepping a source tree for an
identifier is
orders of magnitude more wasteful than simply using an inverted index.
If grepping the tree were a major part of what I do then I'd do
something about it. As it is, it isn't. It also doesn't affect the
end user. If I were running on a slower, less capable machine (or
over a slow network or some other constraint) then I'd have to adopt
a different approach. But none of those constraints hold so it's
simply not an issue so I've not wasted any time trying to develop
anything "better".
Antonio
arcarlini at iee.org
10:37 a.m.
New subject: Performance-oriented development in the commercial computing world
> [...]
developers don't WANT to work on slow machines.
Wouldn't make much
difference to me. [...] 3) Stuff-all else that taxes a machine (the most
demanding thing
would be a web browser and occasional unit test runs).
You don't specify what "a web browser" is. What most people mean by
that is something that often won't run at all, never mind run well, on
"slow machines" like my SS20.
Are there common things that a programmer waits for?
Compiles have
hardly been a burden in the past decade...
Then you're already using fast machines, and, what's more, you're
working only on small pieces of software. A year or two ago, I was
working (for $THEN_JOB) on something that - on a "modern" "fast"
machine, mind you - took something like half an hour to build from
scratch, and annoying long, multiple minutes, even to do a "nothing
changed" build. And that's with everything already in cache.
I moderately often work with a piece of software that, even on a
relatively modern machine, takes significant time - long enough for a
human to get frustrated - to build. I just now did a test build, in a
ramdisk on a Core 2 Duo at 1.8GHz, and it took a minute fifty-four
seconds.
/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTML mouse at rodents-montreal.org
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
11:21 a.m.
New subject: Performance-oriented development in the commercial computing world
On 30/10/11 1:37 PM, Mouse wrote:
That is true. But then, there's lynx :)
A lot of development these days doesn't get compiled (e.g. PHP).
And yes, for big projects, you need to scale up somewhat. But it's not
as if compiling is what one does all day. In between compiles, you need
to do what a programmer does. :)
> [...] developers don't WANT to work on slow
machines.
Wouldn't make much difference to me. [...] 3) Stuff-all else that taxes a machine (the most
demanding thing
would be a web browser and occasional unit test runs).
You don't specify what "a web browser" is. What most people mean by
that is something that often won't run at all, never mind run well, on
"slow machines" like my SS20. Are there common things that a programmer waits
for? Compiles have
hardly been a burden in the past decade...
Then you're already using fast machines, and, what's more, you're
working only on small pieces of software. A year or two ago, I was
working (for $THEN_JOB) on something that - on a "modern" "fast"
machine, mind you - took something like half an hour to build from
scratch, and annoying long, multiple minutes, even to do a "nothing
changed" build. And that's with everything already in cache.
I moderately often work with a piece of software that, even on a
relatively modern machine, takes significant time - long enough for a
human to get frustrated - to build. I just now did a test build, in a
ramdisk on a Core 2 Duo at 1.8GHz, and it took a minute fifty-four
seconds.
One source file? Or many?
--Toby
/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTML mouse at rodents-montreal.org
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
11:51 a.m.
New subject: Performance-oriented development in the commercial computing world
You don't
specify what "a web browser" is. What most people mean by
that is something that often won't run at all, never mind run well,
on "slow machines" like my SS20.
That is true. But then, there's
lynx :) A lot of development these days doesn't get
compiled (e.g. PHP).
Well, doesn't get compiled in the sense we're talking about here; I
think PHP, and certainly many of the "doesn't get compiled" languages,
do indeed get compiled, if only to pcode, when run.
And yes, for big projects, you need to scale up
somewhat. But it's
not as if compiling is what one does all day.
Sure feels like it sometimes - which I suppose is what I was saying.
In between compiles, you need to do what a programmer
does. :)
This is very true.
I moderately
often work with a piece of software that, even on a
relatively modern machine, takes significant time - long enough for
a human to get frustrated - to build. I just now did a test build,
in a ramdisk on a Core 2 Duo at 1.8GHz, and it took a minute
fifty-four seconds.
One source file? Or many?
4:28 p.m.
New subject: Performance-oriented development in the commercial computing world
On 30/10/11 2:51 PM, Mouse wrote:
There is. lynx is what I use when I have occasion to care about
something Web (which isn't really all that often; I avoid the Web in
general). (That's why I qualified it with the "what most people mean
by that" part. :)
Have you ever waited for that to happen?
Many. git. (There are worse - X, and NetBSD itself, come to mind -
but I don't mess with them often enough to feel they're fair examples
for this discussion. Though come to think of it it's been a while
since I did anything to git....)
Yes, if you are a programmer working on Xorg or NetBSD kernel, you may
want a reasonably modern machine.
If you are not, then you compile them infrequently :)
You don't specify what "a web browser"
is. What most people mean by
that is something that often won't run at all, never mind run well,
on "slow machines" like my SS20.
That is true. But then, there's
lynx :) A lot of development these days doesn't get
compiled (e.g. PHP).
Well, doesn't get compiled in the sense we're talking about here; I
think PHP, and certainly many of the "doesn't get compiled" languages,
do indeed get compiled, if only to pcode, when run. And yes, for big projects, you need to scale up
somewhat. But it's
not as if compiling is what one does all day.
Sure feels like it sometimes - which I suppose is what I was saying.
In between compiles, you need to do what a
programmer does. :)
This is very true.
I
moderately often work with a piece of software that, even on a
relatively modern machine, takes significant time - long enough for
a human to get frustrated - to build. I just now did a test build,
in a ramdisk on a Core 2 Duo at 1.8GHz, and it took a minute
fifty-four seconds.
One source file? Or many?
I think the largest single source file I work with may be my tar, which
is 304672 bytes, 11301 lines (I just checked). I just test-compiled
that on my SS20, and it took 28 seconds - without benefit of ramdisk.
How often do you modify and recompile that file? If you changed it every
day, would you refactor? Would there be any other benefits to breaking
it into modules?
--T
/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTML mouse at rodents-montreal.org
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
8:57 p.m.
New subject: Performance-oriented development in the commercial computing world
A lot of development these days doesn't get
compiled (e.g. PHP).
Well, doesn't get compiled in the sense we're talking
about here; I
think PHP, and certainly many of the "doesn't get compiled"
languages, do indeed get compiled, if only to pcode, when run. [T]est build,
in a ramdisk on a Core 2 Duo at 1.8GHz, and it took
a minute fifty-four seconds.
One source file? Or many? If you are not, then you compile them infrequently :)
Actually, that does not follow. I build NetBSD significantly more than
I actually change it. This is for two reasons: (1) when I change it, I
build it on multiple machines (all the machines which run the affected
version), and (2) I do a build when installing a new machine even if I
haven't touched it since last time around.
However, it's reasonably likely that I still build them infrequently,
for the value of "infrequently" you intended. :)
I think the
largest single source file I work with may be my tar,
which is 304672 bytes, 11301 lines (I just checked).
How often do you modify and
recompile that file? If you changed it every day, would you refactor?
I don't know. Possibly, but also possibly not. Depends on too many
other factors not stated in the question. I don't really have any
perceived "I really oughta break this thing up" need, so...
Would there be any other benefits to breaking it into
modules?
...I see no great need. There probably would be other benefits; I'm
not sure whether the benefits would be worth the prices.
/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTML mouse at rodents-montreal.org
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
12:44 p.m.
New subject: Performance-oriented development in the commercial computing world
On 10/30/2011 10:27 AM, Toby Thain wrote:
I think a lot of the problem is much less "work" related than that.
Most developers I've met want super-fast development machines at work so
they can play games, watch videos, and do all sorts of other crap that
has nothing to do with actual work.
Yes, I've fired a few of these types of people. With the American
work ethic of late, though, they seem to be in the vast majority.
-Dave
--
Dave McGuire
New Kensington, PA
> [I]t's
not because it's "stupid" (which it isn't), it's because
> developers don't WANT to work on slow machines.
Wouldn't make much difference to me. Because, at least when I do it, my
activities are:
1) A *lot* of thinking
2) A little bit of typing
and
3) Stuff-all else that taxes a machine (the most demanding thing would
be a web browser and occasional unit test runs).
Are there common things that a programmer waits for? Compiles have
hardly been a burden in the past decade... The machine should nearly
always be waiting for the programmer while they think about what they
are doing. ("More thought, less code" is a whole other rant.)
I'd be equally productive on a 10 year old box; but I'd have the benefit
of knowing if I were doing something silly, performance wise. A poorly
phrased database query in a web app will be a problem on the fastest
server, as client load increases. And judging by the bloatware around,
examples surely abound in client side development. How can fast testing
machines not hide performance problems? Performance/resource bugs are
bugs too. 
25 Oct
25 Oct
2:35 p.m.
New subject: Performance-oriented development in the commercial computing world (was: Fixed point financial data versus floating point)
> An easy cure is give developers slow machines
with limited memory and
> slow disk subsystems. I do that to myself. I find it a good way to
> discover the "right way" TM
On Tue, 25 Oct 2011, Richard wrote:
This is a recipe for expensive software. I've
seen this idea floated
verbally more than once, but any company that ever cared about
performance or memory footprint (and there are plenty) never did
this, and for good reason. It makes your software ridiculously
expensive because you cripple the productivity of every person
working on it. You're better off developing automated benchmarks
against your software that test the performance or footprint sizes
and having those benchmarks fail with a noisy report as soon as
someone steps across the "that's too much" threshold.
I have long held the position that the biggest problem with most modern
software, PARTICULARLY Microsoft, is that they have grossly indequate
TESTING on slow, small, flaky hardware. When queried, they often state
that they see no need for that. "That's a hardware problem".
"Upgrade
your hardware". "I'm surprised that it will even RUN on such obsolete
junk (6 months old!)" There is an implication that YOU are inadequate if
you complain about performance on machines that are not TODAY's issue.
SOME hardware problems are rare but inevitable, such as disk I/O errors on
read and write. PROPERLY written software should be so crash proof that
it can report what happened and exit "gracefully" when they occur. But,
will that be written into the software if the "developer" has never
personally encountered a hardware error? Will the "developer" do an
adequate job of exception handling if they have never experienced running
the software at less that SICTEEN TIMES the "required" capacity and speed?
Yes, developer productivity is up with state of the hard machines.
BUT, too many current "developer"s do not have a reasonable level of
experience with slow, small, or flaky hardware!
IFF there were adequate testing on lower grade hardware, then certainly
the bulk of the development could be done with high-end machines. BUT,
THERE IS NOT.
If there is not going to be multiple levels of hardware available to
facilitate proper testing, and an obsession that all machines in the
development process should be the same, then they would, indeed ALL need
to be downgraded to lowest level to get proper testing.
If most of the people in the world aren't doing
what you think is
smart, then perhaps you should look deeper into the situation before
declaring the rest of those people idiots. Maybe they know something
you don't.
and maybe they don't.
Microsoft, for quite some time (since the days when Bob Wallace walked
out), has not had an appropriate testing protocol. My favorite example
was the implementation of write-caching in SMARTDRV (1991). They refused
to listen to Beta testers bug reports, because "It works fine on OUR
computers", even when presented with analysis of the problem and that use
of the software on flaky hardware WOULD result in data loss that would, by
the very nature of the problem, end up being blamed on OTHER Microsoft
products. Accordingly, "disk compression" got an undeservedly bad
reputation, and MS-DOS 6.00 had to be upgraded AT NO CHARGE from 6.00 to
6.20. ALL due to failure to test on lower-grade hardware.
"Developer"s with experience on lower-grade hardware don't make THAT kind
of mistake.
There is a normal tendency to pick fan-boys for Beta testing. Microsoft
is not unique in that. I am not really surprised that I was dropped from
their beta test program after I submitted a well documented and thorough
bug report ABOUT THEIR BETA PROGRAM.
--
Grumpy Ol' Fred cisin at xenosoft.com
2:53 p.m.
New subject: Performance-oriented development in the commercial computing world (was: Fixed point financial data versus floating point)
On 25 Oct 2011 at 14:35, Fred Cisin wrote:
There is a normal tendency to pick fan-boys for Beta
testing.
Microsoft is not unique in that. I am not really surprised that I was
dropped from their beta test program after I submitted a well
documented and thorough bug report ABOUT THEIR BETA PROGRAM.
Some of the most surprising test outcomes can come from clueless
users, who don't know how the software works, but who have an idea of
how it *should" work.
They could crash software that had been through the most devious
tests that T&I could devise. Some of it was a simple as turning off
the system in the middle of transaction...
The other thing you learned was that users know what comprises a good
UI much better than the engineers do.
--Chuck
3:55 p.m.
New subject: Performance-oriented development in the commercial computing world
On 10/25/11 21:35, Fred Cisin wrote:
While I agree strongly with heavy testing on underfeature hardware, I do
know of one case where the opposite condition exposed the bug - actually
poor programming pracitce. Back in the early years of IBM virtual menory
deployment, we received a program that the vendor claimed ran perfectly
during their tests. We would get results ranging from totally "off the
wall" answers to abends. When we dug into the dumps, we found a lot of
random values in various data areas.
Turns out that the developer was running on a system that was thrashing
its vm pages, so each memory request was receiving a zeroed memory page.
We had just upgraded to a system that was sized to support our growth
over 3-5 years. Most of our jobs didn't even page, and when the
application was loaded, whatever was in the memory page from the
previous jobstep was still present. As far as the OS was concerned the
same "owner" got the memory, so clearing the pages wasn't a security risk.
The cause of the issue was that the programmer, seeing that he always
had zeros during early testing, only initialized NON-ZERO data elements.
It took us nearly a month to convince the vendor that there really was a
problem, with many post-mortem dumps snailmailed back and forth.
--
larry bradford
> An easy
cure is give developers slow machines with limited memory and
> slow disk subsystems. I do that to myself. I find it a good way to
> discover the "right way" TM
On Tue, 25 Oct 2011, Richard wrote:
This is a recipe for expensive software.
I've seen this idea floated
verbally more than once, but any company that ever cared about
performance or memory footprint (and there are plenty) never did
this, and for good reason. It makes your software ridiculously
expensive because you cripple the productivity of every person
working on it. You're better off developing automated benchmarks
against your software that test the performance or footprint sizes
and having those benchmarks fail with a noisy report as soon as
someone steps across the "that's too much" threshold.
I have long held the position that the biggest problem with most modern
software, PARTICULARLY Microsoft, is that they have grossly indequate
TESTING on slow, small, flaky hardware. When queried, they often state
that they see no need for that. "That's a hardware problem".
"Upgrade
your hardware". "I'm surprised that it will even RUN on such obsolete
junk (6 months old!)" There is an implication that YOU are inadequate if
you complain about performance on machines that are not TODAY's issue.
SOME hardware problems are rare but inevitable, such as disk I/O errors on
read and write. PROPERLY written software should be so crash proof that
it can report what happened and exit "gracefully" when they occur. But,
will that be written into the software if the "developer" has never
personally encountered a hardware error? Will the "developer" do an
adequate job of exception handling if they have never experienced running
the software at less that SICTEEN TIMES the "required" capacity and speed?
Yes, developer productivity is up with state of the hard machines.
BUT, too many current "developer"s do not have a reasonable level of
experience with slow, small, or flaky hardware!
IFF there were adequate testing on lower grade hardware, then certainly
the bulk of the development could be done with high-end machines. BUT,
THERE IS NOT.
If there is not going to be multiple levels of hardware available to
facilitate proper testing, and an obsession that all machines in the
development process should be the same, then they would, indeed ALL need
to be downgraded to lowest level to get proper testing.
If most of the people in the world aren't
doing what you think is
smart, then perhaps you should look deeper into the situation before
declaring the rest of those people idiots. Maybe they know something
you don't.
and maybe they don't.
Microsoft, for quite some time (since the days when Bob Wallace walked
out), has not had an appropriate testing protocol. My favorite example
was the implementation of write-caching in SMARTDRV (1991). They refused
to listen to Beta testers bug reports, because "It works fine on OUR
computers", even when presented with analysis of the problem and that use
of the software on flaky hardware WOULD result in data loss that would, by
the very nature of the problem, end up being blamed on OTHER Microsoft
products. Accordingly, "disk compression" got an undeservedly bad
reputation, and MS-DOS 6.00 had to be upgraded AT NO CHARGE from 6.00 to
6.20. ALL due to failure to test on lower-grade hardware.
"Developer"s with experience on lower-grade hardware don't make THAT kind
of mistake.
There is a normal tendency to pick fan-boys for Beta testing. Microsoft
is not unique in that. I am not really surprised that I was dropped from
their beta test program after I submitted a well documented and thorough
bug report ABOUT THEIR BETA PROGRAM.
--
Grumpy Ol' Fred cisin at xenosoft.com
26 Oct
26 Oct
6:29 a.m.
New subject: Performance-oriented development in the commercial computing world (was: Fixed point financial data versus floating point)
At 02:51 PM 10/25/2011, Richard wrote:
This is a recipe for expensive software. I've seen
this idea floated
verbally more than once, but any company that ever cared about
performance or memory footprint (and there are plenty) never did
this, and for good reason.
Amen. Give the developers the best environment you can afford.
If you want to test and benchmark on slower machines, that's fine.
- John
25 Oct
25 Oct
12:56 p.m.
New subject: Performance-oriented development in the commercial computing world (was: Fixed point financial data versus floating point)
> An easy cure is give developers slow machines with
limited memory
> and slow disk subsystems.
There's an Apple person I share an IRC channel with. He says that
developers have been using typical-consumer-spec machines rather than
grossly overmuscled "developer" machines recently, specifically to
dogfood their stuff properly.
I still am in awe at the speed of the average personal
computer.
[...64K loop...]
Of course, it didn't matter. The analysis finishes
near-instantaneously. Running a truth table with 16K entries
through Espresso also completes in almost no time at all.
I'm clearly of the "old' mindset.
Yeah. Me too. I recently dipped a toe into evolutionary programming.
This involved "running programs" many, many times over. The task was
game playing, so this means running games with multiple individuals as
players. My population consisted of 10000 individuals and I wanted to
run ten times that many games before starting to evolve, in order to
have some data to work with.
This means not just a loop that iterates 100K times, but a loop that
iterates 100K times with the body of the loop running, on the average,
about two individuals' programs.
I ran this on my fastest machine, of course (one core of a 1.2GHz Core
2 Duo). I was expecting it to take substantial time, as humans measure
time - probably somewhere between a few seconds and a few minutes.
It didn't. It finished so fast I had to check to verify it really did
do what it was supposed to.
But "take care of the pence and the pounds will take care of
themselves". When I code, efficiency is always, at the very least,
lurking at the back of my mind; I went thorugh my larval stage on a
VAX-11/780 and some of the habits it produced will probably always be
with me.
I don't consider this a bad thing. Any hardware can be made to crawl
by throwing a sufficiently large problem at it, and the more
efficiently the code runs, the larger the problem required to do so.
And, to tie it back into the list topic, it means I can, and usually
do, write code that does useful stuff on classic machines.
"Old" mindset FTW!
/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTML mouse at rodents-montreal.org
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
5:48 p.m.
New subject: Performance-oriented development in the commercial computing world
On 25/10/11 3:01 PM, Chuck Guzis wrote:
On 25 Oct 2011 at 18:15, Dave Caroline wrote:
We know this because you did not optimise unnecessarily.
The problem today is at least two-dimensional. As Mouse lamented, there
is no frugality or understanding of the great excess of speed and space,
in order to use it effectively; but the salt in the wound is a
widespread obsession with *micro*optimisation... time and attention
wasted looking in the wrong directions.
This is clear from forums everywhere - for example, the person who
recently asked in a C/C++ channel which operator was "faster", < or <= ;
or those who ask which integer datatype is "faster" in a SQL schema; or
those who refuse to contemplate languages with high level abstractions
suiting the problem at hand.
These questions, like Tony's "can't find 382.73 Ohm resistors", betray a
truly frightening lack of understanding of what is being done.
--T
An easy cure is give developers slow machines
with limited memory and
slow disk subsystems. I do that to myself. I find it a good way to
discover the "right way" TM
I still am in awe at the speed of the average personal computer.
When programming the analysis software for the PAL "cloning" project,
I was concerned about running nested loops inside of an outer loop
that iterated 65K times (PAL16L8 chips for example, have some tricky
aspects, such as tristate outputs and feedback lines), remembering
how long a simple 65K decrement-and-jump sequence took on a 4MHz Z80
(about a second, if memory serves).
Of course, it didn't matter. The analysis finishes near-
instantaneously. Running a truth table with 16K entries through
Espresso also completes in almost no time at all.
I'm clearly of the "old' mindset.
--Chuck
5:40 p.m.
New subject: Performance-oriented development in the commercial computing world
On 25/10/11 1:15 PM, Dave Caroline wrote:
On Tue, Oct 25, 2011 at 5:10 PM, Chuck Guzis<cclist
at sydex.com> wrote:
I fully agree.
--Toby
On 25 Oct 2011 at 10:50, Mouse wrote:
An easy cure is give developers slow machines with limited memory and
slow disk subsystems. "The most amazing achievement of the
computer software industry is its
continuing cancellation of the steady and staggering gains made by the
computer hardware industry." - Henry Petroski
I remember an Intel engineer summarizing this more succinctly
somewhat like this:
"We work hard to build chips with amazing performance and the
software people piss it all away."
--Chuck
I do that to myself. I find it a good way to discover
the "right way" TM
Dave Caroline
5:46 p.m.
New subject: Performance-oriented development in the commercial computing world
Dave Caroline wrote:
On Tue, Oct 25,
2011 at 5:10 PM, Chuck Guzis <cclist at sydex.com> wrote:
On 25 Oct
2011 at 10:50, Mouse wrote:
"The most amazing achievement of the computer
software industry is its
continuing cancellation of the steady and staggering gains made by the
computer hardware industry." - Henry Petroski
I remember an Intel engineer summarizing this more succinctly
somewhat like this:
"We work hard to build chips with amazing performance and the
software people piss it all away."
4:23 p.m.
New subject: Performance-oriented development in the commercial computing world (was: Fixed point financial data versus floating point)
On 25 Oct 2011 at 10:50, Mouse wrote:
> "The most amazing achievement of the computer software industry is its
> continuing cancellation of the steady and staggering gains made by the
> computer hardware industry." - Henry Petroski
On Tue, 25 Oct 2011, Chuck Guzis wrote:
I remember an Intel engineer summarizing this more
succinctly
somewhat like this:
"We work hard to build chips with amazing performance and the
software people piss it all away."
Hence my corollary to Moore's Law:
. . . and the software will double its inefficiencies to compensate.
24 Oct
24 Oct
12:11 p.m.
On Mon, 24 Oct 2011, Vintage Coder wrote:
Decimal math (hardware supported) is used heavily in
financial
processing with IBM COBOL. No loss of precision because the type is base
10. BCD is very similar to what IBM calls "packed decimal".
Although with a radix of 10, which MAKES SENSE for financial, I still
think of that as being scaled integers. It is not an exponential
structure like the IEEE Floating Point Representation Standard.
1:53 p.m.
On 24 Oct 2011 at 12:11, Fred Cisin wrote:
Although with a radix of 10, which MAKES SENSE for
financial, I still
think of that as being scaled integers. It is not an exponential
structure like the IEEE Floating Point Representation Standard.
...and there's always CBASIC, PASCAL/M, SuperCalc, etc. There was a
mindset (seems to have largely faded) that a program was no good for
finance unless it could boast decimal computation.
--Chuck
4801
days inactive
4808
days old
88 comments
20 participants
participants (20)
-
alexeyt＠freeshell.org -
arcarlini＠iee.org -
cclist＠sydex.com -
cisin＠xenosoft.com -
dave.thearchivist＠gmail.com -
fraveydank＠gmail.com -
geoffr＠zipcon.net -
gyorpb＠gmail.com -
jfoust＠threedee.com -
jhfinedp3k＠compsys.to -
la_bradford＠cox.net -
legalize＠xmission.com -
madcrow.maxwell＠gmail.com -
mcguire＠neurotica.com -
mouse＠Rodents-Montreal.ORG -
ploopster＠gmail.com -
spectre＠floodgap.com -
teoz＠neo.rr.com -
toby＠telegraphics.com.au -
vintagecoder＠aol.com