7 letter words and 8 card suits

A recent commenter asked whether the following is true: "ETAERIO is the most likely seven-letter word to get in scrabble."


As all scrabble players know, if you use all 7 of your letters, you get a bonus. However, except for the first turn, you'd need an 8-letter word to achieve this--ETAERIO would not do. Thus, I am going to try to answer the question: "What are the chances of getting the letters ETAERIO on your initial turn in scrabble (you also have to hope you are first)?" I have no hope of finding out whether this is the most likely seven letter word, because I can't automatically check all letter combinations, but I will try to give some guidance there as well.


To find the chances of gettting ETAERIO, we need the number of combinations that produce these letters divided by the total number of combinations. In other words, we have to go back to 12-th grade math, where we all learned (or sort-of learned) permutations and combinations.


There are 100 tiles in scrabble and we are choosing 7. Thus, there are 100 ways to to choose the first tile, 99 ways to choose the second, and so forth down to 94. If we chose them in order, we'd have 100*99*98*97*96*95*94 permutations. However, we don't care about the order, so we have to take the above product and divide by the number of ways we can permute the 7 tiles, which is 7*6*5*4*3*2*1. The shorthand way to express this number of combinations is "100 choose 7" or




By dividing the two ratios above ((100*99*98*97*96*95*94) / (7*6*5*4*3*2*1)), we come up with 16,007,560,800. Since most letters appear multiple times, the number of possible letter combinations is far less, and to know the chances of getting ETAERIO, I need to know how many times each letter appears.



Thus, I found the letter distributions on Wikipedia (counting our own scrabble pieces would probably not do with a three-year old distributing them around the house). The most common ones as follows:
E - 12 tiles
A, I - 9 tiles
O - 8 tiles
N, R, T - 6 tiles
D, L, S, and U - 4 tiles
other letters - 3 or less, but not relevant here


To figure out the chances of getting ETAERIO, we need to know the number of combinations that produce it. We need 2 E's, 1 T, 1 A, 1 R, 1 I, and 1 O. It turns out that the number of ways is the product of each of these implied combinations. Thus, it is "12 choose 2" (E's) times "6 choose 1" (T) times "9 choose 1" (A) and so forth. This comes out to 1,539,648 ways to get the letters in ETAERIO. If we divide this by the total number of combinations (16,007,560,800), we find that there is about a 1 in 10,000 chance of getting ETAERIO as your first 7 letters. Of course, from there, you have to know it is a word and figure out that you can make that word from those letters, since they are not likely to appear in that order.


I could not find another word with a higher probability, but I did find TREASON and TRAINED (both about 1 in 20,000). However, It's clear from the distribution of letter tiles that in order to find a word that beats ETAERIO, you can only use letters appearing in 6 or more tiles.


BRIDGE HANDS

Now that we all remember the mechanics of combinations (or at least, we are on the subject of them), let's investigate another oft-asked question around here: what's the chance of being dealt a 7 card suit in bridge? This would be 4 (number of suits) times "13 choose 7" (ways to choose 7 from a suit) times "39 choose 6" (ways to choose the other 6 from the other 3 suits) divided by "52 choose 13" (ways to choose 13 cards from 52). This comes out to about 3.5%, or 3 or 4 times in every 100 hands.

For an 8-card suit, it is 1 in about 200. For a 9-card suit, it is about 1 in 2,700. Of course, my kids are always asking about the chances of being dealt a 10 card suit or even a 13-card suit:
10-card suit: 1 in 60,738
11-card suit: 1 in 2,746,693 (less than 1 in million)
12-card suit: 1 in 313,123,057 (less than 1 in 300 million)
13-card suit: 1 in 158,753,389,900 (less than 1 in 150 billion)

The chances aren't too great, but with some really poor shuffling, they've managed the 13-card suit once or twice.

Rent or Buy?

The answer to the age-old question, according to every grandmother out there, is "buy." But do the data really support this?

Housing as an Investment
Forgetting for the moment about the psychological advantages and disadvantages of buying versus renting, let's look at the Economics, and, of course, the probabilities.

One of the most fascinating pieces of information to answer this question is a chart put together by Robert Shiller (Yale Economist) as part of his book Irrational Exuberance (he also has an article on the housing market with similar charts in Economists' Voice, March, 2006). Shiller looks at inflation-adjusted housing prices from 1890 to the present. I got the chart from this site, and the whole article is free and downloadable here (search on Shiller).



Shiller sets the price of a house in 1890 at 100, and shows how the value varies over time, adjusting for inflation. Thus, in 1947, soon after the war, the value is 110, 10% higher than in 1890. In 1989, the peak of the last boom, it is around 125. And now? Around 200!


Besides the obvious "irrational exuberance" of the housing market that is indicated in this graph, another interesting fact comes out: Housing goes up and down over time and in any given 20 or 30 year period, can be either a good investment or a bad one. Sure, if you timed the last couple of booms correctly, you could have made a killing, but the fact is that a house bought in 1960 was basically the same price in 1995, after accounting for inflation. Of course, the person who bought that house could have lived there for 35 years, paying only the cost of upkeep, and, presumably the mortgage.

Total Return to Renting Versus Buying

That brings us to the next topic: knowing that a house may or may not give you any real capital appreciation, is it better to buy or rent?

One of the big arguments I hear from my mother-in-law against renting is that "you are just throwing your money away." Seems like a good point. With renting, you get absolutely nothing out of it, but with buying, after 30 years, you own a house. The problem with this argument is it ignores two things: 1) the down payment, and 2) interest.

When you buy a house, you put around 20% down. That money then cannot be invested elsewhere. In addition, you pay interest on a loan, whose proceeds are invested in the house. The good thing is that you are using the proceeds from your loan to buy an asset 5 times the value of what you invested in cash. For example, if you buy a $500,000 house, you only have to pay $100,000. Thus, if you are in one of the boom times in Shiller's graph, you get 5 times what his graph shows in return on your $100,000 investment. The flip side, of course, is that in the bust times, you get 5 times the losses.

Compare this to renting. Here, you keep your $100,000, perhaps investing it in safe 5-year Treasury notes, where you can expect an inflation adjusted return of about 2.5% (see the fed site for T-note rates and the Bureau of Labor Statistics site for inflation rates--the real return is lower if you go back more than 40 years).

Now, let's look at renting or buying with specific numbers. Suppose that you have a $500,000 home in mind. For buying the house, your total costs are your mortgage and your return is the amount you get back after the sale, minus the $100,000 you paid as a down payment. For renting, your costs are your rent and your return is the cash you get in interest from your $100,000 investment.

The following table lays out 6 scenarios (click on the table to see a legible version). I've adjusted for the tax benefits of the mortgage as well as inflation.

I varied the mortgage interest rate and the (annual) house appreciation. The two numbers at the bottom are your out of pocket monthly costs after 5 and 10 years for owning the house. Presumably, this is the amount in rent you should be prepared to pay. Thus, for a house that appreciates at the rate of inflation (roughly what has happened with housing since 1890--it is the 3% appreciation in the 2nd and 5th columns), your monthly costs are $633 if you have a 6% mortgage and sell after 5 years and $1,134 if you have an 8% mortgage. You do much better if you hold the house for 10 years. Not true for the house that does not appreciate. Then, you do better if you hold it for less time.

To really answer the question well, we'd need an accurate prediction of inflation and mortgage rates. In the short term, both of these are pretty easy. The second thing you need is an idea of how long you will hold the house. If the house appreciates at the rate of inflation, then you are better off holding onto it for longer. If it does not, then you are better off selling sooner.

What is apparent is how radically the value changes depending on the assumptions about how much the house will appreciate. Put in some negative numbers and it really gets scary. If the current boom results in a 2% annual nominal depreciation over the next 5 years, then your 5-year monthly cost goes to about $2,500. At 5% a year depreciation (something that occurred over several years with NYC apartments in the early 90s), then your out of pocket is $3,500 per month if you sell after 5 years.

So, should you rent or buy?

Well...it depends.

Throw away your cold medicine?

A recent article in the New York times touts: "Seawater Seems to Beat Medicine in Fighting Colds." The article goes on to describe a study where "scientists assigned 289 cold or flu patients ages 6 to 10 to be given a nasal wash three times a day with water from the Atlantic Ocean that had been commercially processed but retained seawater’s trace elements and minerals. As comparison, a group of 101 children used ordinary over-the-counter cough and cold medicines."

The Times gets this first part wrong. The "seawater" group got both standard medications and seawater, as explained by the journal article on the study. So, right away, we are not talking about throwing away our cold medicines. Instead we might have to buy (for those of us not close to the ocean) seawater (more below about saline vs. seawater).

But the study does have some interesting results.

Here's the good part
The results for the preventative success are most striking: at week 12, 25% of the children in the control group had reported illnesses that caused an absence from school versus only 8% in the treatment group. This is statistically significant, meaning the results were too large to be explained away by mere chance. This does not mean, however, that biases in the study could not have caused the difference (no matter how statistically significant, bias, if it exists, can mean an otherwise statistically significant difference is spurious).

Here's the bad part
1. The study was not blind. This means that the children (and physicians and parents) were aware of whether the kids were taking the saline solution or not, subjecting the study to a "placebo" effect: kids who were taking the saline might have 'felt' better, but had no less incidence of a cold. The study's authors make an error in the journal article by stating: "the large number of participants, multi center design, and consistence of results between individual parameters (assessed by physician, patient, and parent) lower the risk of bias."

Bias is not mitigated by sample size -- that is, a large biased group is no better than a small biased group (imagine trying to figure out average height of all men by taking an NBA team, then doing a second study with the average of all NBA teams, saying this lessens the bias).

Similarly, having three biased parties (physician, patient, and parent) would only reduce the bias if we were comparing it to the bias of the most biased party (say, the parent).

2. The treatment is no fun.
Perhaps as important as the questionable effect of the study due to bias is the fact that the treatment involves the solution being squirted into the kid's nose 3 times a day for 12 weeks. I was sort of amazed that the study had so few dropouts (only 11 out of 401 patients in the study dropped out). The unpleasantness seems barely worth the avoidance of a cold or two.

3. Seawater, water or saline-- does it matter? The study does not provide any comparison of the seawater spray to other sprays, or even a simple water spray. To its credit, the journal article does not focus on the fact that the solution was seawater but instead on the comparison of nasal wash versus no nasal wash. In the journal article, there is little indication that seawater would be any better than salt water (the word seawater is mentioned 10 times in the journal article as opposed to saline, which is mentioned 64 times). BTW, the seawater in the study was processed (and presumably sterilized), so don't take it literally and go to your neighborhood polluted beach for your solution.

The Times article, however, focuses on the idea of seawater, as opposed to a simple saline (salt-water) solution.

Net Net
It does seem that washing your nose out with saline 3 times a day will make your kids feel better and they will miss less school. But it's unclear whether they are actually any less sick or they just think they are less sick (to that end, maybe giving them a sugar pill each day, telling them it was a special cold pill, would have the same effect).

Election Math - Update

So my guess was wrong. Clinton won. The question is, was it the various selection biases and undecided voters, the measurement error (polls were on Saturday and Sunday but the election is on Tuesday), or something else?

Unfortunately, there is really no way to tell. However, there are two interesting things to note.

One is that in one local poll, they show the percentages both including and excluding those leaning toward a candidate but are undecided. In this case, Obama received 2% more of the vote if the leaners are counted, indicating there is some bias in the method that most polls use, which is to count the leaners (who are actually still undecided) rather than exclude them (the polls do exclude the truly undecided, which has hovered in the 5-10% range, but implicitly assume they will vote the same way as the decided voters).

The second thing to note is that the polls very accurately predicted Obama's percentage and inaccurately predicted Hillary's. Obama's actual percentage was 36%, and the seven major polls predicted 36%, 35%, 39%,41%,34%,38%, and 39% (an average of 37%). Hillary's percentages (same web page) were 28%, 34%,28%,28%,31%,29%, and 29% (average of 30%) versus her actual of 39% (a difference that is outside of the zone for mere statistical error). Hillary apparently picked up votes from undecideds, people who were leaning for Obama, and from the Edwards/Richardson camps.

Election Math

Today's CNN headline screams out "Obama opens double-digit lead over Clinton." When you read the article, you find that the poll, of 341 Democrats, showed Obama over Clinton, 39% to 29%. This is as compared to a Saturday poll showing them in a dead heat, at 33-33 (http://www.presidentpolls2008.com/ is a nice polling site, because it shows the polls side by side and you can link to the details of the statistical error and actual questions asked).

Did so many people change their minds in one day? If we ignore other polls, then the statistical evidence is not conclusive. Why? Because the margin of error in the 39-29 poll is 5%, meaning that the Obama's percentage is likely somewhere between 34 and 44%. The Saturday poll, also with a 5% margin of error, indicates his percentage is between 28 and 38%. The overlap between these two ranges means the numbers might not have changed at all. Rather, the difference is mere statistical error, which is an artificat of the sampling. By luck of the draw, the Sunday poll may have found more Obama supporters, even though no one changed their mind.

When comparing two polls taken independently (as above), the error is more than the stated error (5% above) but less than the sum of the stated error in the two polls (5%+5%=10% above). We can compute the error of the difference in these two polls as 7%, which means that Obama's numbers, which appeared to go up by 6% (from 33% to 39%), may have gone down by as much as 1% or up by as much as 13%.

Some math: The 7% is computed as the 1.96 multiplied by the square root of the sum of the squared standard deviations of the polls. The standard deviation of the poll is the error rate (5%) divided by 1.96, or about 2.5%. In a standard probability distribution called a Normal Distribution, 95% of the data falls between plus or minus 1.96 standard deviations from the mean. Thus, in the latest poll, the mean for Obama was 39%, with a standard deviation of 2.5%.

Several implicit assumptions are made in computing the error rate in these polls, primarily summarized as: 1) the Normal Distribution is appropriate, 2) the sample is a random sample of all who will vote in the Tuesday Democratic primary, and 3) the answers in this poll are reflective of the way the voters will actually vote come Tuesday.

Assumption Review
1) Normal Distribution. This one is easy. For a large, random sample and a multiple choice question (who will you vote for), this assumption is always close to reality except when the number polled is very small or the percentages (as are those for say, Kucinich) are close to 0% or 100%. For Clinton and Obama, there is no real issue here, since the sample size is moderately large and their percentages are in the 30-40% range (a neat demo that shows how close a distribution is to Normal, depending on the sample size and percentage, is at http://www.ruf.rice.edu/~lane/stat_sim/binom_demo.html).

2) Random Sample. This is more difficult. Suppose Clinton voters go to chuch on Sunday, followed by lunch, while the Obama voters are home-bodies. This problem can be called selection bias. If there is church-lunch/home-body selection bias, then, in the Sunday poll, a random dialing of phone numbers would have surfaced more Obama voters and would not have been a random sample of Tuesday's voters, as opposed to Saturday, where you might have gotten more equality. [There is generally a second underlying issues of refusals--people who refuse to be polled. If these voters are more likely to vote for Clinton, then Clinton's numbers will be under-stated, but this would be true for both polls that we are comparing.]

3) The difference between how people said they felt in today's poll versus how they will vote in Tuesday's election. I find this difference, which can be called measurement error, the most troublesome. Take, for example, the fact that, in the CNN Saturday poll, one-fourth of voters stated that they had not yet decided, another quarter were only leaning toward someone, and just half had definitely decided. This is, of course, only what people are saying, and often people do not want to admit indecision, so the true numbers of undecided may even be higher. Still, if the undecided's vote even 60-40 in favor of hillary, it would erase the 10% lead of Obama.

The 5% error rate (and 7% error of difference rate) does not take the above issues into account. It implicitly assumes they will have no effect. Thus, the true error rate in election polls is likely far higher.

If we consider other polls, the Obama lead, and the change seems to be clearer. In the seven polls published th 6th of January, Obama has an average lead of about 2-3%. In the 5 polls published Friday and Saturday, Clinton led in all of them, by around 5 points. We can prove this change, from pre-Sunday to Sunday, is statistically significant. However, because of the selection bias and measurement error issues above, it may not be indicative of the outcome on Tuesday.

My personal guess? Obama by a good margin...but a lot can happen in a day.

Where is the safest place to live?

If you figure out where to live by the crime rate, many of the safest places to live are outside the United States, where crime, though lower than it used to be, is still high by Western standards. Here in Israel, for example, fewer than 200 people are murdered a year. In the U.S., about 17,000 people were murdered. Of course, these figures are not comparable due to the fact that Israel has around 6 million people as compared to the U.S.'s 300 million.

Crime rates and murder rates are typically adjusted for population by reporting the number of crimes per 100,000 people. In Israel this rate (for murders) is about 3, as compared to about 6 per 100,000 in the U.S. Even including terror attacks, the rate has never been as high in Israel as in the U.S. In England, the rate is less than 2 per 100,000, which is in line with most of Western Europe.

All this is fine and good if you are willing to live abroad just to enjoy a lower crime rate, but, assuming you are looking for a nice place in the US to live, what city might suit your needs vis-a-vis lack of crime?

I considered the four cities I have lived in (see the FBI site for all the stats), Columbia, SC; Washington, DC; Phildalephia; and New York. Of these, New York (sorry Mom) is clearly the safest, with a murder rate of 6 per 100,000 and a violent crime rate of about 0.7% (less than 1 in 100). The worst is DC (murder: 29, violent crime 1.5%), but Philadelphia is a near tie (26, 1.5%). Columbia, my home town and my parents' current and future town, is in the middle (13 murders per 100,000 and 1.1% violent crime rate).

These stark differences mask three important facts. First, the chances of a random individual being a victim of a violent crime in any given year, is very low, no matter what the city (though it's more likey to get killed by a person than a car in Philly and DC, according to National Safety Council statistics--originally linked here but apparently not on the web anymore as of at least 11/28/2012).

Second, most violent crimes are committed by someone we know, and most of the people reading this blog do not have violent acquaintances, plus most murders are committed with guns, and most people reading this blog do not have acquaintances with guns.

Third, crime is highly localized, and no matter which city we live in, we are likely to live in a more affluent area, and these areas have much lower crime.

So, in conclusion: don't worry, Mom, New York is safer than Columbia, as long as I can keep making a living!

The lottery: a tax on stupidity?

It is often quipped that the lottery is a tax on stupidity (google "lottery: tax on stupidity" and you will see what I mean). I've always been bothered by this for two reasons: 1) one of the first people who said it to me was an arrogant professor who I always enjoy proving wrong, and 2) someone in my family (who has an advanced degree in mathematics) used to (and may still) play the lottery. So I figured there must be some fallacy in this statement.

I use New York's Lotto as an example, because it is a very simple "jackpot" lottery game. In this game, you choose 6 numbers between 1 and 59 (inclusive). If all 6 match, you win the jackpot. The order of the numbers does not matter. To figure out the chances of a match, you must compute the number of equally likely possible combinations (or just go the Lottos website and skip the next paragraph).

To compute these chance, we first figure the total number of combinations of 6 numbers out of 59. Suppose we choose the numbers in order. Then, we have 59 choices for the first number, 58 for the second number, and so forth until we have 54 for the 6th number. This totals 59*58*57*56*55*54=32,441,381,280 permutations in all. Since we do not care about the order, however, we need to adjust this number (consider, for example that 11,12,13,14,15,16 and 16,15,14,13,12,11 are both the same set of numbers and considered the same in the Lotto drawing). This adjustment is made by dividing by the number of possible orderings of six numbers, which is 6*5*4*3*2*1 = 720. Thus, divide 32,441,381,280 by 720, and you get 45,057,474--the number of possible combinations, of which you choose two for each $1 Lotto ticket. Your chances of winning, then, are about 1 in 22.5 million. The average jackpot is about $9 million (this jackpot amount is only obliquely referred to on the NY Lotto website, in that they say that 40% of revenues go to the jackpot).

Given these odds, how much do you expect to win if you buy a single ticket (good for choosing two six-number combinations)? Well, given the odds of 1 in 22.5 million, you would clearly expect to win absolutely nothing!

But mathematicians don't think this way. Instead, they compute the expectation as the long run average, and by long-run, I mean really really LONG-run (actually infinite-run, but let's not split hairs). To give you some idea of this, you would need to play around 15 million times to have a 50% chance of winning at least one time--this would take 41,000 years or so if you played 1 ticket a day. So, computing the average after a few thousand or even a few million games is likely to get you an average of 0, which is *not* the correct long-run average.

Instead, this expectation is computed by taking the sum of the probabilities of winning multiplied by the amount won. In the case of the Lotto, then you win $0 in (22,499,999/22,500,000) games and $9,000,000 in (1/22,500,000) games. So, the Expected winnings are(22,499,999/22,500,000) *$0 + (1/22,500,000)*$9,000,000 = 40 cents.

So, you pay a dollar, and "expect" to get 40 cents back. This is why some people call the lottery a tax on stupidity. When people say the lottery is a tax on stupidity they are implictly and incorrectly assuming that utility (to throw in an economic term) is based purely on mathematical expectation, and that the utility from $9 million is 9 million times the utility from $1. Yet I doubt that people are playing the lottery based on some mis-guided mathematical expectation calculation. $1 or 40 cents. Who cares? Either way it's barely worth picking up off the ground.

Smart people who play the lottery are valuing 2 things against each other -- $1 versus a miniscule chance of $9 million -- and deciding that the value of $1 to them is less than the value of the chance at the $9 million. Yes, poor people probably value $1 more than average, but they value a chance, even a small one, of forgetting about their financial woes even more.

Let's look at another game that shows the flip-side of this mathematical expectation conundrum. For all you upper-middle class, non-lottery players out there, consider the following: Would you pay your entire net worth for a 1 in 1,000 chance to win $10 billion? If your net worth is less than $10 million, this is a game with positive expectation. For those of us with less than $1 million hanging around the house, the expectation is more than $9 million, but I doubt you'd find any middle-class person willing to play this game.

Why? Because the risk is too great, no matter what the reward. It is widely recognized that people place different values on risk. Risk averse people are willing to lose a small amount of money (or pleasure) to insure they will not lose a large amount of money (or pleasure), even when the mathematical expectation of their transaction is negative. The best example is insurance (Wikipedia's lottery entry points this out). Insurance companies make money not on stupidity but on the fact that people do not want to take large financial risks.

So next time you hear someone say the lottery is a tax on stupidity, tell them about the mathematician who plays, or about the people who turned down a game with an expectation of $9 million.