Why Building Expensive New Apartments Makes Old Apartments Cheaper

I often hear from people that I normally agree with on other issues that supply and demand don't apply when it comes to housing. I don't know why people who study housing all say one thing, and a corner of the advocacy space says another. I do have experience with Berkeley housing markets (albeit from the 1980s) and that soured me on rent control as the sole method to manage scarcity of an essential good. 

Berkeley has long had a rent control ordinance and a rental registry to ensure compliance. Jeff Baker wrote some Python code to visualize the public data and it is fascinating. As a scientist, I always like to see real world data compared to idealized models of how things work. 

The Rent Board of Berkeley, California maintains an Online Rent Registry with information about apartments and other dwellings under regulation by the city's rent control laws. The Registry lists the initial rent of a rent-controlled dwelling, and all of the subsequent annual adjustments.

The Registry tells us a bit about the city's rental market over time. Using the initial rent of tenancies still active at the present, we can deduce the market price of apartments in the year the tenant moved in. The chart below gives the median initial rent each year 1996-2022, adjusted for inflation to 2022 U.S. dollars, for tenants who are not listed as owners, managers, or any other special class.

Go to his site and read his analysis.  

One surprising thing I learned is that, though rents in rent-controlled units are generally lower than in newer buildings not subject to rent control, they do rise unevenly instead of monotonically rise the maximum amount allowed. Rent-controlled rents plateau during recessions. 

The population of apartments in the above figure are all older buildings, due to the nature of rent control in California. The rents shown are significantly below the advertised rents of the same size apartment in a new building in Berkeley. For example, some new buildings are currently advertising 1-bedroom apartments for around $3000 and 2-bedroom apartments at about $4200. The rents in old, regulated buildings are discounted by about 1 bedroom.

So he took the initial rents for newly-signed leases, and deflated them by the CPI from BLS All items in U.S. city average, all urban consumers, seasonally adjusted using a constant 2022 dollar (when I think he wrote/started this project). 

There's a lot of hand-wringing about how today's building code requiring two staircases results in mostly studios and 1-bedroom apartments with a few 2-bedroom units on corners. So, new construction is heavily weighted towards smaller units. Not too great for families, but fine for 1-2 person households. 

Older units subject to the most stringent rent control have to report the new leases within 30 days of signing the lease. Newer buildings only have to report annually, Each dot represents the average rent for initial leases signed in one month. Note that the dots are really tiny, meaning hardly anyone moves. In a university town of 100,000+ population, less than 20 people move into 1-BR rentals. 

In Dec 2017, 18 new leases for 1-BR apts were signed at an average rent of $2267 (2022 dollars). A bunch of new apartment buildings came online in downtown Berkeley, allowing people to move. That is when the bubbles become larger. By Dec 2024, 27 new leases for 1-BR apts were signed with an average rent of $1950 (2022 dollars). At the same time that the rent stock got more numerous, more modern (new units), it got cheaper in real dollars. 

Remember, older rent-controlled units have to report monthly. Newer buildings can report annually. The bigger circles are from the months that the newer buildings reported en masse, so the average new rents for those months are higher.  In August 2023, a whopping 785 new leases for 1 BRs were signed.

Notice how the small circles are falling faster than the big circles? 

What's going on here? I heard that building new, expensive apartments makes rents in older units more expensive! 

The data in Berkeley shows that new, expensive apartments makes rents in older units decline! Score 1 for the housing economists who say that supply and demand applies to housing. 

There are fewer new 2-bedroom apartments because of the International Building Code (IBC), but the same effect is noticeable in the 2-BRs. 

In Dec 2017, 14 new 2BR leases were signed for an avg of $3124. 

In Dec 2023, 18 new 2BR leases were signed for an avg of $2529. 

In Aug 2023, 275 new 2BR leases were signed, which is less than the 785 new 1BR leases for the same month. (Thank-you IBC.)

IBC hardly ever results in new 3 BR homes. But, when you have too few smaller apartments, people move in with roommates and outcompete families with children for the larger units. Even without building new 3 BR homes, just allowing people living with roommates who would rather live on their own (or in smaller households with fewer roommates), causes the rent for larger homes to stop sharply rising. 

In June 2017, 19 new 3BR leases were signed for an avg of $4534. 

In June 2023, 110 new 3BR leases were signed for an avg of $4728. 

The circles also get larger, meaning that there are more vacancies that allow people to move to something that better suits their needs and desires. That's a win for overall happiness. 

So, rent control provides certainty for incumbent renters, but doesn't allow them mobility if they need a larger or smaller unit. It also makes it impossible for new entrants to the market to find housing. 

Creating new homes, even expensive ones (IBC is expensive to comply with), can lower the rents on older units. 

The economists were right in this instance. 

CA Car Rebates and Our Underfunded Active Transportation Program

There's been much hoopla about California's budget surplus and high gasoline prices. So why not use some of that surplus to alleviate pain at the pump? That may be good political messaging when the impetus is the much more mundane Gann Limit on CA public spending. The ghost of Howard Jarvis strikes again. Not content to limit just property taxes, they sponsored and got the electorate to approve caps on overall spending that limit public investment overall.

I don't want to belabor the stupidity of giving people who own cars $400 per car, up to $800 per person, while not similarly rewarding people who are either too poor to own cars and/or care enough about the common good to not own a private car in the first place.

In a world without all the stupid laws that we inherited, we could have fully funded our Active Transportation Program (ATP) to increase the proportion of trips accomplished by biking and walking.  The majority of ATP-funded projects are Safe Routes to Schools--to help children get safely to and from school. Basically, we need to protect kids outside of cars from the cars chauffeuring them around. 

Because of limited funding, ATP grants are extremely competitive. 

In 2014, cities and counties across the state requested about $1 billion in funding for pedestrian and bicycle safety projects, but there was only $368 million available, meaning about 37 percent of applicants were funded that cycle. Fast forward to Cycle 5 in 2020 when over $2.5 billion in funding requests were submitted for $554 million in available funding, a success rate of about 22 percent. In Los Angeles County, only 14 of 64 applications were awarded even partial funding, or 22 percent total – demoralizing, yet consistent with the statewide average.

The 2023-24 ATP budget is even grimmer. ATP has $147,670,000 to spend that is not already committed to other projects. That means, the 6-county SCAG region of 20 M people (including LA County, has only $31,242,000 or about $1.50/resident. 

In contrast, Governor Newsom's proposed 5-year infrastructure plan will devote $10 B to electric cars and $20 B for roads, roughly $5 B/year (pages 7-8).  This isn't even counting the $ spent on CHP and traffic enforcement. Due to the Gann limit, every $ spent in one place is a $ we can't spend somewhere else. This is extremely discouraging. 

In the mean time, we depend on volunteers and advocates such as Safe Routes Partnership to help communities hone their proposals to improve their odds of winning an ATP grant. "In ATP Cycle 5, four out of the five communities we worked with scored an 86/100 or above." In other words, communities can compete to get technical help to further compete to get funds to improve street safety for school children. 

My community finally won an ATP grant, but the funds allotted are well short of what we really need to remodel our streets.  We're likely to end up with some paint and street signs. Sigh. 

We have so much work to do. Spend some time exploring the California ATP Transportation Injury Mapping System.  (You need to register to create a free account, but it's worth it. UC Berkeley researchers built the system and don't do anything nefarious with your search terms.)

Here's a heat map of the 2017-2021 carnage. 

People who live in the neighborhoods with the larges blotches of red are least likely to own a car but most likely to be killed or maimed by one.  In Los Angeles County, over 5 years, 173 cyclists dead, 1323 pedestrians dead, thousands more injured and maimed. Their lives will forever be marked by pain and disability. (I'm not even counting the effect of air pollution in their neighborhoods.)

The Gann Limit requires CA to give out rebates. I wish that the rebates be used for restorative justice instead of rewarding people for owning cars. Who's with me?

New Construction Subsidizes Old

I was reading city council minutes and my head was going to explode with all the coded things that some council members were saying. In one discussion, Council Member NN wanted to impose Quimby Fees on the theory that newcomers have not been paying taxes into the community and should have to buy their way into the parks and other amenities of this city. That is, they didn't pay for the park purchases so they shouldn't get enjoyment until they pay their debt to existing homeowners. 

This is so nonsensical, because, he then talked about using the Quimby Fees to buy new parks or to maintain existing parks. That sure sounds like he wants new construction to subsidize old homes, not the other way around. Hmm. Coincidentally, he wants nearly all the new housing (and Quimby Fee payers) next to the freeway in the extreme NE corner of the city while using their Quimby money to purchase parks in the coastal southern area of the city that he represents. 

BTW, the southern end of our city has been identified as a Racially Concentrated Area of Affluence. In a county that is 26% non-Hispanic white, he represents census tracts that are currently over 80% non-Hispanic white (and formerly whites-only on HOLC "red line" maps.)

Back to this idea that new construction doesn't pay their fair share for infrastructure, let's take a look at this screenshot from the California Property Tax Viewer. 

Take a look at the mixed use building with 48 homes, 8 affordable (hence, lower green property tax flags). There is also a garage and several businesses, generating more property and sales taxes since 2011.  200' by 120' = 24,000 sf. One condo unit is paying $7,200/yr in property taxes. 

Look at the single family home (SFH) behind it on a 50' by 150' = 7,500 sf lot.  It pays only $1,300 in property taxes each year or about 18% as much as one of the 48 condos. It was built in 1953 so it's been paying property taxes for 67 years.

Let's do a back of the envelope calculation and just use ratios.  One ten year old condo has already paid as much in property taxes as the older house has in 55 years.  Plus, our city charges Quimby Fees of $25,000 per net new home and a bunch of other fees totaling about $29,000.  That adds up to another 22 years of taxes from the old house. 

And that thing about sewer lines?  Complete nonsense.  Our water and sewer mains run down the street and maintenance costs go by the frontage.  That old house sits between a 6-plex on one side and a duplex on the other, each on the same 50' wide lot.  

Our sewers are run by the LA County Department of Sanitation and we're charged per hookup.  The infill townhomes are paying 2-6 times as much to maintain the sewers as the old SFH. The mixed use development has about 50 hookups on 200' so they are paying 12.5x as much for our shared infrastructure.  

The water and infrastructure myth is even more pernicious in light of water conservation (which is a good thing!) Read Adapting to Change: Utility Systems and Declining Flows. As we conserve water, the residence times in the water pipes increase, sometimes too long to be safe.  Infill housing, if it adds more people, can help keep the water residence times safe.  

If you don't add enough infill housing, you end up in San Diego's fix.  In just a few years, the cost of expensive potable water (imported from the Colorado River) they had to flush to keep their domestic water safe has increased from $200,000 to $2,000,000/year. 

Speaking of flushing, water-saving appliances and toilets mean that sewage outflow is lower volume and thicker than in the past.  That requires more energy to pump it back to the sewage treatment plant.  If managed well, infill housing can offset declining flows, save energy and reduce clogging.

Infill housing helps in two ways.  It gives you more customers to share the cost of maintaining infrastructure. And it also offsets declining flows for both inbound potable water and sewage.

That SFH is the deadbeat in the picture.  Long live infill!

LTN: One possible solution to decarbonizing transportation

Bad Dad and I were featured in A Local Travel Network for the South Bay Story Map* riding my Ebike and an Escooter we bought for combining with transit just before the March 2020 lockdown.

Click through to see the full wide picture that also includes a BMW electric car.  I have mixed feelings about this project.  I think it is a reasonable baby step, but the South Bay Cities Council of Governments does not appear inclined to offer more than sharrows.  In fact, their South Bay Bicycling Master Plan counts roads with 50+ mph traffic and a sign on the shoulder saying it is a bike route as a bikeway.

The Story Map makes some good points about the South Bay region of Los Angeles County.  This area is home to roughly 1 Million people and 750,000 cars.

70% of South Bay trips are less than 3 miles, yet we do most of them by car. It's both a problem of habit and the built environment. We don't provide safe spaces for people out of cars, so people make trips in cars, even if they would prefer to do otherwise. Spend some time exploring pedestrian and cyclist data using UC Berkeley's Transportation Injury Mapping System. 

In 2009-2020, there were 100,000 collisions involving pedestrians and cyclists, including ~3100 deaths,  in Los Angeles County.

This is just the people that braved the streets outside of cars. This doesn't even represent the suppressed active tranportation trips that people took in cars or forwent out of (quite rational) fear.

I'll take allies where I find them.  I am accepting the LTN (if they actually build it) as a down payment, but not as payment in full, for the safe streets that we deserve and need as we decarbonize local transportation.

* We were told to be near the intersection of Pacific and 10th street in Manhattan Beach, CA one Saturday morning to film.  We bike through that area on our way to the beach several times a week.  It's one of the most expensive neighborhoods in the region.  The organizer wanted a quintessential South Bay setting with the ocean.  But, I would not have selected a neighborhood that aggressively protects Single Family Home Zoning to preserve the affordability of $30,000,000 ($30 Million) dollar homes.  We have to quit showing SFHs as if they are normal or representative. Nothing could be further from the truth.

Can flying ever go green?

Define green.

Today I learned that United Airlines uses biofuels from municipal sewage for a portion of their jet fuel for flights out of LAX.  In 2016, they contracted to purchase up to 15 Million gallons of biofuel over 3 years.  At the end of the contract, they signed up to purchase up to 10 M gal/year, doubling their use.

In 2016, WaPost wrote that the biofuel would be blended in a mixture of 30 percent biofuel and 70 percent traditional fuel.

But the use of biofuels is one possibility for existing machines to cut down on their emissions without having to upgrade their engines or other aspects of their design or engineering. “Drop-in” fuels are renewable fuels that are designed to work safely with existing engines, although as in the case of the United flights, they sometimes require mixing with traditional fuels.

This is analogous to cars in California using a blend of 15% bioethanol and 85% gasoline. If the ethanol comes from grains like corn, then the carbon savings is minimal.  If the bioethanol and biomethane comes from food waste, then the payoff is bigger.

Putting it in perspective.  United Airlines used 4292 Million gallons of fuel in 2019, of which 5 M gal was biofuel.  Even if they were to double that to 10 M gal/year, that would be 0.2% of their fuel use.

So, biofuels in airplanes is better than diesel.  But it's even better if we cut down on our flying overall.  It doesn't have to be much.  Cut down on one round trip long haul flight per year and it's as much carbon savings as if you traded in your gasoline-powered car for an electric car.

I copied the figure from Quantifying the potential for climate change mitigation of consumption options, which I also wrote about in New Paper on Global Data-Driven Climate Actions.

We cut back on personal flights so that we make one long-haul trip every few years.  I was supposed to fly home from Germany yesterday-today.  Bad Dad was going to represent his family at the commemoration of the liberation of Bergen Belsen 75 years ago.  The ceremony will be postponed.

We don't fly long distances for short trips.  We make fewer, longer trips.  We had planned to spend 3 weeks, visiting Berlin, Munich and some smaller towns in between.  I even planned to show him the small town where I had been an exchange student in high school (German Gymnasium.)  It will have to wait.

Big Data, Big Planet

I wrote Big Data, Big Planet for UCARConnect explaining what I do to K-12 teachers and high school students.

Climate reconstruction with atmospheric rivers, including the one that flooded one third of Los Angeles in 1938.

The November 2016 issue of UCARConnect is all about Data: The Currency of Science.

Animation from The atmospheric river that caused the Los Angeles flood of 1938.

#sewphotohop Day 27: other interests

It's day 27 and I'm running out of steam for #sewphotohop.

One look at the word cloud in the column at right shows that I have a lot of interests.

When I started this blog, I had to be coy about my day job because I worked in a military lab, and the security officers really, really did not want their scientists to be out in public. It's not that I was doing anything to be ashamed of. In fact, I really enjoyed the work I did there. It's just not in the culture there to bring attention to oneself.

Since 2014, I've been providing data support for climate and weather research.  We are an open access data provider, funded by the National Science Foundation.  Both the data and the data tools are provided free thanks to governments around the world.  If you pay taxes (and I hope you do), then you help pay for this data infrastructure.

All this is an excuse to provide a list of data links.

• 'Weapons Of Math Destruction' Outlines Dangers Of Relying On Data Analytics
• Digital Forensics Rescues Retro Video Games And Software
  

  It takes a particular personality to spend one’s life feeding the NSRL* . You would need the passion of a collector, the sensibility of a curator, the technical skill of a computer scientist, and the ability to find satisfaction in a job that you know will never be done.

• Meet Carla Hayden, the First Woman Sworn In As Librarian Of Congress (LOC)
• She spent much of her career here: Baltimore’s Enoch Pratt Free Library Provides Haven in Troubled Times
• Type "Carla Hayden Patriot Act" into a search engine to learn why I am so excited about her appointment.
• She's not only the first woman and the first African American, but she's also the first leader of the LOC appointed in the internet era and who also understands data.  She's also way more qualified than the white guy who was appointed in 1987 by Ronald Reagan and who totally botched the transition to the digital era.
• Her PhD dissertation: "A Frontier Of Librarianship: Services For Children In Museums." University of Chicago, 1987 combines two of my favorite institutions, museums and libraries!
• Today, LOC hosted Collections as Data: Stewardship and Use Models to Enhance Access. Click the link to view on YouTube.
• Some of the techniques to crowdsource reading digitized US civil war records are similar to ones used by the International Environmental Data Rescue Organization to digitize and read old climate logbooks from around the world.

I have a passion for making stuff.  I have a passion for data.  I have a passion for sharing my knowledge and skills.

*National Software Reference Library, a division of the National Institute of Standards and Technology (NIST).  I received my PhD at JILA, another part of NIST.

More statistical nonsense

I'm bereft that Serena lost today.

There's also the matter of a presidential candidate suggesting or joking that people assassinate his opponent and the judiciary.

Let's talk about something that makes me mad, but only mildly so.  This also gives me a chance to jump up and down on my soapbox about bad data crunching.

Exhibit A, this piece of click bait from the NY Times with a tone of schadenfreude toward engineering majors:

I clicked and read these counter-intuitive numbers.

This doesn't jibe with my personal experience. Physical scientists that I know are very, very civically engaged. How could we be such slackers when it comes to getting to the voting booth when I see "I voted" stickers on everyone in lab on election day?

Do I know a very atypical set of physical scientists?  I had a hunch that, perhaps, it is because (outside of school and student jobs) I have always worked in national labs that require US citizenship?

I did a little research.

First, I went to The National Study of Learning, Voting, and Engagement (NSLVE) website and read about the project. There appears to be a database accessible from that website. Because I'm not a participant in the research, I lack access to it.

There also appear to be some scholarly articles, which might have the summary data cited by the NY Times.  Again, I lack access to the articles.  (I'm not going to pay $41 for 24 hours of access to an article that may or may not have the data I seek.)

Search for "The National Study of Learning, Voting, and Engagement report". I was able to find several, including reports for Columbia and Long Beach Community College students.

In each report, I saw that the figures for % of eligible students voting by major was calculated using IPEDS and the same percentage was applied to all majors at a school, given the schools' overall demographics.

This is based on the percentage of non-resident aliens reported by your institution to the Integrated Postsecondary Education Data System (IPEDS), and is more reliable than the demographic data campuses provide to the Clearinghouse at this time.

Do you see the statistical flaw? The reports gave the numbers with this caveat at the top:

Your students broken down by field of study. Please note that we are not able to adjust these voting rates by removing non-resident aliens.

The NY Times' poorly-researched and reported listicle did not include any methodology or context.

OK, now let's read what the National Science Foundation has to say about Higher Ed in Science and Engineering.

• About 60% of all foreign graduate students in the United States in 2010 were enrolled in S&E fields, compared with 32% at the undergraduate level.
• Foreign students earned 57% of all engineering doctorates, 54% of all computer science degrees, and 51% of physics doctoral degrees. Their overall share of S&E degrees was one-third.
• In 2009, temporary visa students earned 27% of S&E master's degrees, receiving 46% of those in computer sciences, 43% of those in engineering, and 36% of those in physics.

Moreover, physical science and math students are vastly outnumbered by business and other students; the US graduated 19 business majors for every math or statistics major in 2011.

Let's list what we know:

• Statistics tying individual students majors and voting behavior are difficult to obtain for privacy reasons.
• They had to make estimates based upon school-wide statistics.
• Each school reported the % of their students that were not on temporary visas.
• NSLVE then applied the same % to all majors, even though they know this is inaccurate. They reported that this is a source of error.
• They also removed students that were younger than 18 and not eligible to vote.
• The % of students studying STEM is quite low compared to other majors, particularly business.   That gives larger error bars to STEM voting numbers, even without the eligibility estimation.
• STEM students as a whole make up ~20% of the total undergraduate (UG) population, but 30% of the foreign UG student population; their voting participation is underestimated by the NSLVE methodology.
• This means non-STEM students are more likely to be US natives; their voting participation is overestimated by the NSLVE methodology.
• Foreign-born permanent residents are a wild card.  They do not need a temporary visa.  Yet, they cannot vote.  They are also disproportionately likely to be studying STEM.
• Foreign students make up a disproportionate share of STEM students at every level, but particularly so at the graduate level.  They dominate in many STEM fields.  Thus, their voting participation is VASTLY underestimated by the NSLVE methodology.  (That 40% of physical science students could very well be 90% of eligible students.)

I found all sorts of interesting information, especially at the National Center for Education Statistics:

• Degrees conferred by sex and race
• Most popular majors I double majored in two of the most unpopular majors!
• Table 318.20. Bachelor's, master's, and doctor's degrees conferred by postsecondary institutions, by field of study: Selected years, 1970-71 through 2011-12
• Table 322.10. Bachelor's degrees conferred by postsecondary institutions, by field of study: Selected years, 1970-71 through 2013-14
• NSF: Chapter 2. Higher Education in Science and Engineering
• Voting Behavior Among College Students Broken out by gender, race, political ideology, school selectivity and Freshmen vs Seniors.

Anyway, after examining the data, I think it is very, very likely that physical science students that are eligible to vote do so at higher rates than journalism students.  I'm sure The average physical science student is better with data than the average journalism student.  We might even be better than the average NY Times journalist.

Another piece of bullshit debunked.

Good-night.

Data Thinking Before Data Crunching

Years ago, I was a data analyst and subject matter expert.  Now that Data Scientist is the job description du jour, I rebranded myself.

Rebranding really works.  My online resumes generated a lot more views, though I remained substantively the same person, with the same skills, education and experience.

Lemmings.

Anyway, my pet peeve is how fixated interviewers are about how much data you've crunched, instead of how well you did it.

I really, really, really like my job as a data specialist and educator.  In case you wonder why a forty-something married mother would move away from her family for a job that barely pays enough to run a second household and fly home, I thought I would show you a sample of my work.

I posted my slides and notes for a recent talk, Data Thinking Before Data Crunching, given to a mixed audience of students, working software developers and scientists-data providers.

I help people use data optimally with the (IMHO) most pressing big data use case facing the world--weather and climate.

It's a good thing that the computer-based work (and my management) allows me to work from my LA home part of the time.

BMGM gets around

When I started this blog, I was working for a military lab.  The security folks took a dim view of splashing the names of their scientists around the web unnecessarily.  Now that I am working for an National Science Foundation (NSF) lab, that is not a problem.

In case you can't get enough of me, I write under my own name at the NCAR Research Data Archive Blog about data and science.  I also contribute to LA Observed, where you can find me in the Native Intelligence section.

My latest posts at both places:

• The data starts here 2
• Death Valley or Spring's Meadows

The latter one allows me to repost one of my favorite pictures.

Benchmarking fracking

(BMGM has been accused of going into the weeds again when I wrote about Benchmarking government.  No, it's just that, as an interdisciplinary scientist, I see connections that don't occur to other people. Or, perhaps, I became an interdisciplinary scientist because I see connections between seemingly disparate things.)

Did you know that, when you buy a home or a plot of land, you don't automatically buy the mineral rights under that plot? Moreover, the seller of a property does NOT have to notify the buyer that the sale does not include the mineral rights.

Nolo.com explains this in plain English:

As a property owner, if someone told you they were going to start drilling for oil on your land, you’d probably try to kick them off as a trespasser. But wait! Unless you also own the minerals under your land, that someone might have every right to start drilling.

In the United States, mineral rights can be sold or conveyed separately from property rights. As a result, owning a piece of land does not necessarily mean you also own the rights to the minerals beneath it. If you didn’t know this, you’re not alone. Many property owners do not understand mineral rights.

This article will discuss what mineral rights are, how they can be conveyed separately from the land they lie beneath, and whether you should worry about someone else owning the mineral rights under your property.

The loophole that does not require disclosure probably stems from the wild gold rush days when mineral claims records were patchy and often lost.  Sometimes, the owner of the property is genuinely ignorant about the status of the mineral rights of the property they are selling.

However, this loophole is being actively exploited by some of the largest home builders in the US.  For instance, Mother Jones explains how D. R. Horton systematically strips and sells the mineral rights from parcels before they build homes.

They used to disclose, before public awareness of fracking.  But, they stopped disclosing once buyers started asking questions and balking.  For tens of thousands new home buyers, their first indication that mineral rights have been stripped from their property is when a truck carrying a rig rolls into their neighborhood as in the photo below.  All photos and maps courtesy of Fractivist and used with permission.

For most homeowners, this is the first indication they receive that they don't own the mineral rights under their home.

New home development in Colorado with oil and gas wells drilled AFTER the homes were sold.

If you were a homebuyer, wouldn't you like a quick and low-cost way to find out who owns the mineral rights under the home and what kind of mining they might do in the future?

This is an extremely tedious process that requires specialist training and often involves travel to look at old records in many different possible records sites.

This is why Benchmarking Government matters and why I think the media and many government watchers are focusing on the wrong thing.  More people, not just Governor Brown, should perform the data retrieval experiment and request the Division of Oil, Gas and Geothermal Resources to provide information pertaining to their land.

Some of the fastest-developing areas in the country--North Carolina, Colorado, inland California--are the areas with the most potential for mining and oil and gas drilling.  Those home buyers deserve to know before they buy.

In fact, people who think they own mineral claims should have a deadline to prove and register their claims to a publicly-accessible digital database or lose their mineral claims.  The public should be able to search this database without charge.

What do people do in the absence of reliable information? They rely on their government to protect them against deep-pocketed oil companies.

Notice the line between Weld and Boulder counties and where the wells lie?  Politics, not oil, often determine the location of active oil and gas wells.  In Boulder County, registered voters skew 41.6% D to 18.8% R versus 38.4% R to 23.2% D in Weld County.  Guess which county banned fracking and which county government welcomes it (despite homeowner protests)?

Politics, not oil, often determine the location of active oil and gas wells.

Yes, I do think it is odd that the political party that purportedly stands for personal choice doesn't want homeowners to choose not to allow fracking under their homes.

This issue contributes to the 'big sorting' of America.  It's divisive and contributes to longer commutes (and more burning of fossil fuels).

This is why I believe it is so important for government to be able to provide complete and accurate records.  I'm willing to pay the taxes to develop and sustain these services because I want the records to be available everywhere--and not just in the places where private companies think they can make money doing so.

Also, I want uniform categorization rules and true data interoperability, something that is difficult to enforce in the dot com sector.  Because of my work, I've been able to observe that government has a good track record on data interoperability.  In the for-profit sector, companies have financial incentive to 'hoard' their data and to make it difficult for others to use.

Benchmarking government

I'm still processing the terrorist killings around the world right now.  I'll leave discussions about that to people who understand it better than I do.

Right now, I want to shed light on a little corner of the universe that I do know better than most.  Hopefully, the amount of understanding in the world will go up a little bit because of what I write.

I am a data specialist in a geophysical data archive so I follow news about geo-referenced data more than the average citizen.  Actually, I'm a bit obsessed with how data searches work or don't work and why.

This editorial appeared in my customized news feed and I was completely flummoxed by the ignorance displayed by the editorial board of a purportedly top-tier newspaper.

The LA Times Editorial Board was incensed by Governor Brown's request to the Division of Oil, Gas and Geothermal Resources--just days after the governor had appointed their new chief--to supply him with a report on the mineral history and the potential for mineral extraction of his family's ranch in rural California. The editorial said:

It's inappropriate for the governor to call the head of an agency for help with personal business, especially someone he had just installed in the job nine days before. It also was wrong for his aides to follow up with the agency to ensure that there would be a map and other specific information. State employees are paid to do state business, not take care of the governor's personal matters. Brown received his report within a couple of days after he asked for it — an uncommon alacrity in state government — and also received a satellite map drawn up especially for him.

When I read that, I was shocked, but not for the reason the editorial suggested.

I was impressed that Governor Brown, a 77 year-old philosophy major, understood the scientific method and how to apply it to data problems.

Whenever you tinker with a system, you run benchmark tests before and after.  If you install a new chief of a department, you measure his effectiveness by testing response time and job quality for a common task required by the department.  Moreover, you run this test for a case that you know well, so you can assess the accuracy of the results.

Asking for all the info on oil and gas extraction in the past, and potential for the future, for the family farm is a great idea.  His family has owned that land for more than 150 years.  If there had been oil and gas exploration on the land in the past, he would have known about it.

I asked my husband, a field scientist, what he thought of the story.  He said that you always test in an area you know really well, so you can gauge the quality of your measurements, before you go to an unknown area.  So that's two scientists who were impressed with the governor's grasp of the scientific method.

The governor got the correct answer in 24 hours, according to this later story with more details.

The wire service story said that "after a phone call from the governor and follow-up requests from his aides," the regulatory agency "produced a 51-page historical report and geological assessment, plus a personalized satellite-imaged geological and oil and gas-drilling map" of the area.

You know, just like any ordinary citizen would expect to receive.

But the characterization of the service appears to be a stretch. Except for a one-page personal memo, all the material collected for the governor amounted to merely a pile of old letters sent other property owners, historic data from yesteryear and some oil field maps.

"Everything is available on the [state] website," said Nancy Vogel, chief spokeswoman for the Natural Resources Agency, the umbrella entity for these regulators. "If you know how to find it.

"They did not do a formal assessment. That would have been many weeks of work."

The governor got back his answer within 24 hours. "The potential for significant oil or gas in this area is very low," the memo read. As for mining, that potential also "is exceptionally low."

Steve Bohlen, Brown's appointee as chief regulator, said the governor asked him about the geology of the land, past oil or gas production and potential for any future production. "I said that was easy to do," Bohlen told me. "It wasn't like 'drop everything.'"

Two petroleum experts who aren't necessarily Brown fans confirmed to me that all this stuff is available on the state's oil and gas website.

That "just like any ordinary citizen would expect to receive," is a low blow. Ordinary citizens in this data-driven era should be able to look up the mineral history of their land (or surrounding land) as that is the best predictor of future mineral development.

The Center for Public Integrity gave California a C- in their 2015 State Integrity report card.  The grade was largely brought down because of an F on Public Access to Information.

Making public information easily available to citizens should be a high priority and the governor should appoint public officials who are committed to improving data processes and data access for citizens.  Running a benchmark test at the start of a new department chief's tenure was the right thing to do.

Let's hope that this media 'gotcha' campaign doesn't deter him from running the 'after' benchmark test to see if they turn up more (or less) data faster (or slower) after Bohlen has been on the job for a while.

Background:

Geophysical data is extremely difficult to search for many reasons.  Records are messy, inconsistent, and often came from the pre-digital era.  So many things can get lost in the translation--or get plain lost.

We are so accustomed to nearly instantaneous searches on the internet, we forgot how much work goes into making this magic mundane.

For instance, do you know how much software and data engineering went into creating this data order form?

Do you know the international treaties that enable the sharing of this data? The small army of people who worked to clean up and standardize this global dataset? It took a whole lot of work to make this look easy. But it was anything but easy or simple.

Related:

I explain why this issue is important in Benchmarking fracking.

Not a sewing video

In case you want to hear my actual voice (not the POV 'voice' you read at BMGM), you can view a tutorial I taped as part of the Yellowstone User Seminar series. Yellowstone* is NCAR's main supercomputer. (The older, retired ones are reserved for use by my section.)

Citizen scientists who don't have an account on Yellowstone may find the general info on how to find and use our open access weather and climate data useful.  In my years as a SAHM and citizen scientist, I downloaded and practiced data wrangling/mash-ups/analysis/science with RDA data.  When they had a job opening, I was in a good position to go pro.

* Since last June, YS has slipped from #29 to #50 on the list of world's fastest supercomputers.  You think supermodels have a short shelf life at the top?  Try being a supercomputer.  Read Slice of Sky.

Wordless Wednesday

I am the harbinger of doom

Bad Dad observed that, if we like a product, that is a pretty good sign they will stop making/selling it. This talent/trait means I have a bit of a hoarder's mentality when I find something I like.

It appears that I am not alone.  In fact, some marketing folks at UPenn have identified consumers who are 'Harbingers' of failure.

I'm the type of person who will be swayed by features like higher quality or compact size, who would buy a Sony Betamax over a VHS.  I would spend $100+ on an iron that lasts 15 years over a $30 one that lasts 1-2 years.  People like me do not rule the consumer marketplace.

"Betavhs2" by Senor k - English Wikipedia. Licensed under Public Domain via Wikimedia Commons.

Actually, I wonder if the study co-authors, Eric Anderson, Song Lin, Duncan Simester, and Catherine Tucker, did not stratify enough. They lump together early adopters with people who buy heavily promoted products "on sale" with people who purchase niche products. I'd like to see if their results hold up if they separate out the three groups.

For instance, if a product appears to be headed for flop status, wouldn't many companies/stores heavily promote the product (on sale!) or close it out (on clearance!).  That lures price-sensitive buyers.  Yet, some of their Harbingers appear to pay more than average consumers, signaling either early adopters or niche consumers.

Anyway, the upshot is that some people have a propensity to select products that are likely to go out of production by the big companies. However, consumers of niche products are also more likely to purchase the products they favor over the internet, and pay more than for bulk commodity products. You don't have to go to Wharton to figure that out. But, what do I know, I'm just a rocket scientist harbinger of doom.

Me Made May 2015 Day 14

I made my top using Vogue 1071 and a remnant of yummy cotton lawn that I purchased from Poppy Fabrics when I was an undergrad at Berkeley. It only took me 20+ years to sew up the piece.

I did not make my pants, socks, or shoes.

This Calvin Klein for Vogue Patterns top takes very little fabric.  I've made this view three times.  I made another view, but, sent it to Goodwill. Poor fabric choice and my inexperience (years ago) with bias edges in rayon crepe made a real mess.

I think that one of the hardest aspects of learning to sew (or knit) is pairing an appropriate fabric (or yarn) and pattern.  What did you find most difficult as a newbie?

In the background, I'm screening Precision and Accuracy in Geodetic Surveying, which uses surveying to teach the difference between precision and accuracy in general.



Lately, I've been thinking and writing a lot about geophysical data formats and metadata standards. To procrastinate do research, I see how others do it.  ;-)

Interpreting the Water Footprint of Food

I've read some discussions of the LA Times Water Footprint of Food interactive feature, mainly expressing surprise about the comparatively large water footprint of pulses, dried legumes.

When you look at these plots, a beef burger looks only slightly indulgent compared to a falafel (chickpeas).  What's the harm in a burger?  Or the Atkins diet in general?

Water footprints of protein sources calculated by the waterfootprint.org.

I scratched my head when I saw that graphic as I understood that beef is a highly inefficient source of protein compared to plant sources (but several magnitudes more efficient than tuna!).  How was this calculated?  Is the methodology valid?

This weather and climate data consultant dug deeper.

The folks at The Water Footprint Network explain their methodology here.  The LA Times article said, "Below you can see the U.S. water footprints of selected foods as measured by gallon of water per gram of protein produced or per calorie."

So is this country-specific data?  How can you compute the water-intensity of the US chickpea crop when there is nearly no commercial chickpea production in the US?

Later, in the LA Times coverage, they quote Mekonnen and Hoekstra:

"The average water footprint per calorie for beef is 20 times larger than for cereals and starchy roots," they note, referring to global averages, not U.S.-specific figures. "The water footprint per gram of protein for milk, eggs and chicken meat is 1.5 times larger than for pulses," a group of legumes that includes peas, beans and lentils.

Perhaps the LA Times reported GLOBAL (not US) water intensity in US units of gallons? The article wasn't clear.

I searched and found that the EU compiled and mapped some statistics they downloaded from McGill University.

Global acreage used for chickpea production.

Global production of chickpeas in tons per square km.

If the LA Times reported US statistics, then the chickpea data is highly suspect.  It's the classic "statistics of small numbers" problem.  The smaller the sample size, the more variable and unreliable the statistic.

Suppose the LA Times did report the GLOBAL water footprint, then it is important to look at the hydrologic cycle of the areas where chickpeas are farmed.

Luckily, I work at a weather and climate data archive and have access to stuff like this classic paper about the terrestrial seasonal water cycle by Willmott and Rowe.  See and download the Willmott and Rowe data.  I give you permission to play with your food data.

First page of Willmott and Rowe.

Recall that chickpeas are mainly grown in India, the middle east and sub-Saharan Africa.  Hmm, look at the evaporation in those regions in the early summer.  If most of the chickpeas are grown in India, and ground-level evaporation is exceptionally high there (and moderately high in other regions where chickpeas are also grown), then the global water footprint for chickpeas will be high.

Evaporation climatology 1950-1979.  Notice the extremely high evaporation in chickpea-producing areas of India and central America during monsoon season. 

Did the LA Times report rely on a small and unreliable dataset (US chickpea production) or report global statistics and label them as US statistics? I don't know. Either way, their reporting is extremely suspect.

People need protein.  Plant-based proteins such as chickpeas are largely grown and consumed in regions where the resources (land, water, labor) cannot support animal sources of protein.

The water and carbon intensity of crops vary greatly by location.  That's why I don't eat an entirely locavore diet.  Our family enjoys CSA boxes grown with reclaimed water from the Irvine waste treatment plant.  But, we occasionally eat lamb chops imported from New Zealand, where the sheep are raised on rainfall-watered pasture.

Yes, sheep meat has a large water footprint.  But New Zealand has abundant rainfall and doesn't need to artificially irrigate their pastures.  If the lamb is frozen and then shipped via ultra-efficient container ships to a harbor < 15 miles from our home, then the carbon footprint of that lamb chop is much, much lower than a beef burger from the Central Valley of CA.

Early summer evaporation in the American west.  Note the hot spot in the Fresno area.

I took a closer look at the evaporation in California.  Download the data and the Panoply data viewer and play with the data yourself.  Scroll through the months.

The Willmott and Rowe data is based solely on 1950-1979 ground-based station data.  Later datasets rely on satellite data, but WR gives monthly averages, which is important because row crops are not grown year-round.

It's important to note that the evaporation measured by the ground stations are influenced by temperature, winds and water availability.  Water availability depends on both natural sources, e.g. precipitation and surface stream flow, and irrigation.

See that bright yellow hot spot near Fresno, California?  100 kg per square meter is 100 mm or ~4" of water.  Look at Fresno's climatology.  That's nearly all irrigation with water from elsewhere or  groundwater.

This is old data.  The numbers today, with millions of acres planted with permanent tree crops that can't be fallowed during droughts, would be even more scary.

This is why California's central valley is sinking.  This is a slow-motion environmental catastrophe.  It has to stop now.

Data as a foreign language

Some say that data is a different language.  I agree that it's a specialized language, often requiring domain or subject matter expertise.

But it still amused me to see that Google Chrome detects this page as Norwegian.

Saying, "yes", to translation did not result in any noticeable changes.

Use this page to access an archive of NASA AIRS brightness temperature data in WMO BUFR format.  Yes, I need to update the software tab to include more recent readers in Java and Python.

BTW,
NCEP = National Center for Environmental Prediction
GDAS = Global Data Assimilation System (DA is the complex process of inserting data into numerical weather forecasts)

Dear Jane

Valentine's day is an appropriate time to tell the story about the most painless break-up ever.  In fact, it was so painless, it was like we never dated.

My first semester of college, I was so grossed out by the food at the dormitory cafeteria menu, I spent a lot of time at the salad bar.  One time, I was surprised to see a male hand piling the raw spinach and alfafa sprouts on his plate.

It's a good thing I spoke to the hand before I followed the hand up the arm to a drop-dead gorgeous guy.  If I had seen him in the entirety, I probably wouldn't have made a friendly remark about the dreadful hot entrees; I'd have been too tongue-tied.

I kept running into him at the cafeteria and he invited me to sit with him and his friends.  It turned out that they were all varsity rowers.  We talked about the normal things that people just meeting each other in college talk about--our majors, home towns, and cultural stuff we were enjoying like books, movies, music, lectures, art shows, etc.  (This was Berkeley.  The football team was terrible and I liked that.)

We also discussed what we had done to deserve a spot at the most coveted dorm.  At that time, UC Berkeley had a student housing shortage.  Students were assigned priority numbers and the more desirable dorms were filled mainly with varsity athletes (him) and academic scholarship students (me).

I learned that he was a bit older than the other students because he had deferred college to travel the world as a print and runway model.  A model scout approached him in high school.  Modeling paid better money than the near minimum-wage jobs that most HS students can obtain.  All he had to do was stay in shape, which he would have done anyway.   Then there was the prospect of international travel.  Why not?

He and his modeling agency had a falling out when he thought it was time to go to college and they thought it was time for him to go to Paris or Milan.  (Models start out in smaller markets to gain experience.  Smaller markets are more diverse and interesting to intrepid travelers; luxury hotels in major cities are more alike.)

One time, he said that he and a teammate were going to rush through dinner so that they could walk down to Telegraph Avenue and look at books.  Did I want to go with them?

Photograph of Moe's Books via Telegraph Ave.

Yes! I love browsing bookstores on Telegraph "Ave". In fact, I used to drive to Berkeley from my suburb as a high school student to enjoy the bookstores of Telegraph Ave. I didn't feel safe walking at night to the bookstores on my own, so the offer of two very large and muscular companions was irresistible.

Photograph via Telegraph Shop

In the following weeks, he sometimes stopped by my dorm room to walk to dinner together.  I did wonder a little why he was passing by my room when his room was on the other side of the cafeteria.  Once, I saw that he wasn't in the cafeteria and went to his room to find out why.  He had fallen asleep after practice and thanked me for making sure he didn't miss dinner.

I remember being surprised when he casually undressed and changed in front of me.  I figured that models must be used to undressing in front of others.  I shrugged it off and we walked to dinner.

A short time later, I found a letter under my door.  He wrote that he was dating two women and things had progressed to the point where he would feel like a heel if he didn't make a commitment to one.  The other woman was older and the femme fatale of the dorm.  They were both older and hot; of course they were a match.

Wait, he was dating two women?  Were we dating?  Did I date a male model and not even realize it?

Years later, I leafed through Let's Go USA and under "Berkeley nightlife" it listed going to bookstores as a popular courtship activity.  I didn't know that my first semester at Berkeley.

A couple of years ago, when I was doing serious self-study about data analysis in the social sciences, I read John Molloy's Why Men Marry Some Women and Not Others.

I was surprised to learn that very attractive men often feel like they are not taken seriously for their intellect.  Many of those men prefer to date and marry very intelligent and ordinary-looking women in the belief that others will project that intelligence to the gorgeous partner.  John Molloy's advice for intelligent women was to approach gorgeous men because we were more likely to be successful in capturing their attention than most women.

So I did date and get subsequently dumped by a male model.  And that's why he would slip in a reference to the scholarship that got me into the dormitory when he introduced me around.

The femme fatale dumped him shortly after that.

When I met Bad Dad, I invited him to go to bookstores with me.  We've been traveling the world and reading together ever since.

Happy Valentine's Day.

* A Dear Jane letter is the female version of the Dear John break-up letter.

Technology Tuesday, Data Edition

I'm the only female data specialist in my department. I thought it was a statistical fluke--our group is small.

But, perhaps, it is a symptom of a bigger problem.

Did you read the Atlantic Monthly's article in which all the data scientists in the photos are men?

The analysis of data is too important to leave to only a thin and homogenous slice of our population. Actually, any job is too important to be left to a homogenous group.

Why?  Because people with different experiences and perspectives can see things that others might miss.  The contributions of a diverse team, testing the data in different ways, without the same assumptions, gives me more confidence in the results.

I read this last night to learn how other women got into data*.  All 15 of them come from the business intelligence or advertising realm.  Their advice range from mindless rah-rah (cheerleading) to the genuinely helpful.  It's short; it's free.

Download your own free copy of Women in Data here.

I work with many female graduate students, post-docs and assistant professors from around the world.  If you need data and data consulting for weather and climate research (that includes journalism and industry), we have a small but very able staff of data experts to help you.  Best of all, our services** are paid for through tax dollars via the NSF (National Science Foundation).

You may not need to pay expensive data scientists to answer your questions.  If you are willing to roll up your sleeves, you can learn to do it yourself.  No super-computer required on your end.

* I explained how I became interested in modeling and data analysis in the introductory chapter of my PhD thesis.  Although I wrote that half a lifetime ago, it still applies.

** There are exceptions for users who need very specialized data products tailored exclusively for them or data rescue services.