This story is about 12M design, but it starts with a simple recent strength discussion. We were looking into the strength of a bollard and lifted some bollard moment strength versus bollard diameter data from the US Navy Towing manual.
To help the discussion, one of our engineers prepared a little spreadsheet and added a quadratic curve fit.
The graph immediately sent me back to one of my earliest big project nightmares.
In 1984 I worked as an engineer for Johan Valentijn. Johan was designing a new 12M for the America’s Cup Eagle Syndicate and was performing model testing at a model basin in El Segundo, California.
Sailboat model testing is very complicated and was first developed by Professor Kenneth Davidson in the 1930’s at Stevens Institute. It was the reason Olin Stephens became a regular visitor there for his America’s Cup designs.
Dragging a powerboat model through a model basin is one thing, but sailboat model testing is order of magnitude more difficult because it requires test runs at a number of speeds, a number of heel angles and a number of yaw angles.
Once that data has been collected one adds assumed sail coefficients (Sail lift and drag coefficients and sail center of effort assumptions, a very tricky issue by itself) and the vessel’s stability (both transverse and fore and aft) to find a steady state condition that satisfies all those forces. If you want to get really fancy, one can also throw in rudder angles, but there are some work arounds for that.
It is not difficult to imagine that a subtle lift to drag, or sail center of efforts change can result in entirely different solutions. In 1982 Peter van Oossanen had proposed and implemented a different method where the model was towed from a stub mast that assumed the center of effort of the sails and then had the model adjust itself in heel and yaw under that load. This looked really good and avoided the cumbersome calculations since it read the solution directly in the model basin, but it relied entirely on the sail coefficient assumption and comparing different models would be difficult because different boats may perform better with different sail coefficients (Full sails, flat sails, etc.)
Van Oossanen used this method to test Australia II and claimed his method was the ticket to better sailboat model testing. There is no strong technical support for that, mostly because it tests the end result, but hides the real data since it combines heeled drag and yawed drag in one solution for his single assumed sail center of effort.
Johan went for the older method and handed me the job of reducing the data. He told me I had to plot all data by hand on all the models he ran, and he also handed me a FORTRAN code listing of a data reduction computer program that had come into use at the various basins. The code listing simply was a dot matrix fanfold printout of the code. It was about 1000 lines long and we actually hired a typist to enter the program lines into an ASCII file to run in the FORTRAN code interpreter and compiler on the IBM XT.
The typist tried her hardest to be accurate, but the printout distinguished between 0 and O just like it is shown here in dot matrix resolution (it did not print the zero with a slash), and she mixed them up randomly. So first we had to figure out which were O’s and which were zeros. And then the program kept crashing because only a minor typo would make it crash. Then there is the realization that there is nothing worse than interpreting someone else’s code, so we graphically mapped the whole program to sort out variable names and subroutine names, and we got it to work.
The input was nice, it was just a simple comma delimited ASCII file.
For each speed it listed the drag at 0, 10, 20 and 30 degree heel and for each heel angle at that speed it listed the drag also at three yaw angles (I forget the actual yaw angles, let’s say 3, 5 and 8 degrees)
If I remember correctly, we did that at 4 speeds. If my math is correct that is 64 runs per model, with lots of model orientation adjustments between runs. Quite an elaborate and expensive testing program and much more complex than non-sailing model testing.
I don’t think we ran through the whole program at different trims, but if I remember correctly, Johan would run the model at different trims to see if there were serious trim induced drag issues.
As a matter of fact, Johan who had done model testing on dozens if not over 100 models over the years, often only needed a few runs on a model to see if it was any good. If it stunk in the first few runs, he would not bother with the full set of runs. (I have done much less model testing, but it is amazing to realize how much just looking at a model running through the water can tell you without even checking the measurements.)
So the computer program used this data and produced curve fits and then would interpolate between those curves to find the steady state solution for different sail coefficients and windspeeds.
When you would do it by hand, you would take the points and hand draw pretty interpretive curves between the points and manually figure out where the forces balanced.
After all that work, I entered the data into the program and got solutions. It was a lot of fun, I could mess with lots of stuff such as vessel VCG, or the fore and aft sail center of effort and see if there were nice optimal results. I even added optimizers for those variables.
But Johan said: “Where are the hand plotted results?”
So I picked a good condition on a model, and did a hand plotted data reduction and, sure enough, it did not match the computer results at all.
I tried again and again, and it did not match. Was there a problem with the program; a coding error? Only those who have lived through one of these issues knows how maddening it can be. Fortunately, the program only needed a few minutes for a run (today it would take much less than a second) and the best way to get an idea about a bug like that is to do lots of “what if” runs.
Nothing showed up and in final desperation I decided to have the program run with continuous pauses and data dumps at each program step. This included data dumps after the curve fits, which I plotted and what I saw was a curve that looked very much like the bollard curve above.
So let’s examine that curve.
The black line connects the data points and the red dots are the quadratic curve fit.
Now if this was a drag curve in a sailboat performance prediction program it would love that spot around 6 where the drag is lowest. Meanwhile, it makes no sense that sailboat drag goes down from 5 to 6 and therefore the performance prediction program returns bad predictions. A hand drawn curve fit would not create such a curve because it makes no sense
Once we discovered that problem within the performance prediction program, we found that our hand drawn curves also had too much room for interpretation. To achieve sufficient accuracy, we added an additional heel angle (15 degrees) to the test program which resulted in curve fits that were sufficiently accurate for performance prediction purposes.
Meanwhile, other syndicates using the same program were claiming performance increases of more than 1 percent and testing at just three heel angles. We knew that they were simply deluding themselves with bad computerized curve fits hidden in a relatively simple FORTRAN program.
This model testing adventure made me very cagey about sailboat performance predictions and in a paper on 12 M design that I presented in 1986 at Stevens, I felt I needed to include a discussion about model testing. I did not want to disclose what we had discovered to our competitors, but made the following remarks:
Historically the towing tank has always received the publicity in yacht design, and it has either been the reason for the success of a design (Intrepid, Australia II) or the reason for the failure of a design (Mariner, Valiant)
I went on to describe the various model testing programs and then noted:
Once a designer has something that appears to be competitive with baseline hulls, he should check the equipment, check his numbers, retest the concept on a different model – ideally retest the concept in a different model basin. ….. Next he should sit down and ask himself whether what he is seeing actually makes sense.
This discussion raised the hackles of the Stevens model basin people when they read the peer review release of the paper. They focused on the word “reason”. I meant the “reason for the publicity”. Apparently, the Stevens people interpreted it as the “reason for the selection of the design and the inference that model testing was flawed”.
I showed up at Stevens with my overhead projector slides, and was rather nervous because there was immense interest in the paper and a large crowd was expected. While everybody was at the bar, I went into the presentation room and made sure the slides were in order and the overhead projector worked. Since it was a large room, the overhead projector was far from the lectern, and I was pondering how I would use the overhead projector and the fixed microphone at the same time. An elderly gentleman walked into the room, and he said: “Are you the young man who will be presenting this paper?”
I said: “Yes, Sir.”
He said: “I am Dan Savitsky, and I will be presenting a discussion of your paper because I have a bone to pick with some of the points you make.”
I was not shocked, but also did not really know what to say, so I said: “Uhm, ok sir, uhm, I noticed that I cannot flip my slides while I am speaking in the microphone. Would you mind flipping my slides?”
He looked at the overhead projector and said:” Sure”, and he walked over to the audience seating pulled over a chair and put it next to the slide projector to make sure he had a chair to sit on when he was flipping my slides. We walked back into the bar area, and in the bar somebody pointed out that the person I was with was Dr. Dan Savitsky the present master of the Davidson model basin and a great hydrodynamicist in his own right. As a matter of fact, I had used his planing powerboat power prediction procedure only a few months earlier.
In later years, when people referred to Dan Savitsky’s work, I enjoyed saying: “Yes, I know Dan Savitsky well. He even flipped my overhead projector slides when I presented a paper on model basin testing at Stevens.”
After Dan was done flipping my slides he presented his discussion of my paper. He had only one slide, so I did not have to flip slides for him. For the first time ever, he published a Velocity Made Good (VMG) curve comparing Courageous and Mariner.
Based on the slide he noted that the basin was not the reason for the failure of a design; the designer (Britton Chance) was the reason. That may be possible, but the next question would be: “Who plotted these curves and how many heel angles were tested?” In his discussion Dr. Savitsky actually noted that these two vessels were tested at the “customary“ 10, 20 and 30 degree heel angles.
This is not a put down of Dan Savitsky. I cannot say Dan Savitsky and I became truly close, but I had several delightful conversations with him over the years and truly respect his great career. However, subtle misinterpretations of data are an ever-present risk even to the greatest experts.
Meanwhile, here, 40 years later, a simple excel curve fit showed up with the same problem.
I want to return to that recent bollard graph. A close observer may note that we did not use a readily available data point, 0,0 which would be a valid data point since a zero diameter bollard can only hold a zero moment. So we reran it with that data point and came up with a curve that is even worse, where there is even a negative maximum moment (or in the case of a drag curve fit; a case of spontaneous propulsion).
Excel allows different curve fits and it turns out a simple power curve is the most useful fit.
But who truly knows?
Don’t for a second believe that such curve fitting (and other) bugs do not exist in today’s sophisticated programs. Always do a quick back of the envelope reality check when using fancy computer programs; if it don’t look right, it probably ain’t right.
And good luck in finding that bug in that computer program when the results make no sense.
