Map of jobs for economists

Posted in economics, visualization on October 31st, 2009 by Michael Ewens – 2 Comments

The AEA’s JOE postings present the near-population of jobs available for newly-minted economic PhDs.  I used the XML data available for download to create a mash-up of job locations on Google Maps.  I break the posting down into US full-time academic, international full-time academic and non-academic.  Here is how I create the maps: Screen shot 2009-10-31 at 9.05.55 PM

  1. Select the subset of the data you want (e.g. US academic) and download the XML file.
  2. Fix some validation errors: take out the “<” and “>” within the text of nodes (I use TextMate for this).
  3. Parse the XML file with a custom PHP script that creates a csv file with school, position, location and url to posting. Here is my simple script for the academic XML file.
  4. Save the csv file produced by the script in step 3 as an Excel spreadsheet (Google Docs doesn’t like csv’s).  Add a “Latitude” and “Longitude” column to the spreadsheet.
  5. Upload the Excel file to Google docs.
  6. Follow these directions to populate the latitude and longitude of each position+location.
  7. Publish the Google spreadsheet and save the unique id in the url that Google gives you.
  8. Sign up for a Google Maps API account.
  9. Follow these directions to produce a Google map of your postings.

UPDATE: This service may make this process a bit easier, produce cleaner maps and allow the incorporation of more information.

Maybe the AEA can follow these directions to produce these maps after this year.  Contact me with any suggestions or questions.

Supply and demand in the jobs market

Posted in economics, news on April 26th, 2009 by Michael Ewens – 1 Comment

From the Indeed job listing aggregation site, one can look at trends in both supply (employers) and demand (potential employees). The demand-side is slightly tricky to interpret because it is a function of not only real employment demand, but the popularity of Indeed as a search platform. Here are some interesting facts:

Finance: Supply of jobs falls, demand increases

Financial Services and Banking job postings have decreased 39% since March 2008.

Clicks on Financial Services and Banking jobs have increased 96% since March 2008.

Construction employment falls

Healthcare bucks the trends

Tax-related employment is volatile

SimplyHired.com also has a trends section with similar patterns.

Also see my own take on apartment rental supply and demand using online listings + web scraper.

Using Mechanical Turk for Research

Posted in economics, research on April 5th, 2009 by Michael Ewens – 3 Comments

Amazon’s Mechanical Turk is a services that allows you to hire 100s of people from across the world to tag photos, complete surveys or find websites. According to Wikipedia:

The Amazon Mechanical Turk (MTurk) is one of the suite of Amazon Web Services, a crowdsourcing marketplace that enables computer programs to co-ordinate the use of human intelligence to perform tasks which computers are unable to do. Requesters, the human beings that write these programs, are able to pose tasks known as HITs (Human Intelligence Tasks), such as choosing the best among several photographs of a storefront, writing product descriptions, or identifying performers on music CDs. Workers (called Providers in Mechanical Turk’s Terms of Service) can then browse among existing tasks and complete them for a monetary payment set by the Requester. To place HITs, the requesting programs use an open Application Programming Interface, or the somewhat limited Mturk Requester site.

Why would economists find this service useful? An example from my own work might help. I am collecting all the individuals employed by new VC firms founded from 1992 – 2007. I need demographic information and employment histories for each VC partner. After scraping the web to get the VC firm websites and “team pages” I have a set of locations for an individual to find the online biography of each of some 3000 VC partners. I submit a job to Mechanical Turk that asks the Turk’er to go to the website, find the individual’s biography and copy and paste the text. Further jobs could ask the Turk to read the bio and answer questions like: 1) Does this person have an MBA or PhD? 2) Male or Female? 3) Founder of firm? Unfortunately, submitting HITs to the Turk system is somewhat difficult. Enter Smartsheet.

Smartsheet as a Frontend to Mechanical Turk

Although the Mechanical Turk service has a system for submitting HITs, it is a little cubersome and requires a bit of strange formatting steps. If you value your time just a little (say, $10/hour) I recommend using the project management webapp Smartsheet’s service SmartSourcing. Again, I refer to my research. I created an Excel file with columns like “Name”, “Firm Name”, “Website” and an empty column “Biography.” I upload this file to Smartsheet, select the rows for which I need biography filled in and walk through their SmartSourcing steps. In about 3 minutes I have submitted a HIT to 1000s of workers that will be complete in 12 hours. I can approve or reject the responses while I watch them populate the online spreadsheet.

Such a service is not free. For a $9.95/month fee (or $99/annual…ask them for a non-profit coupon), you get access to SmartSourcing. Then, on top of the standard Turk fees, you have Smartsheet charges:

Any paid Smartsheet subscriber has access to the Crowdsourcing feature. Monthly charges include Amazon fees plus the cost for work performed (number of tasks completed * cents paid per task) and a low Smartsheet processing fee ($.01 + 10% per task completed – usually $10-$30 per 1,000 tasks).

For example, I paid about $7 for 116 biographies of VC partners. It sounds relatively expensive, but this service has:

  • increased the potential sample size of my studies
  • expanded the set of possible control variables
  • gives you the ability to request multiple workers per task for error checking
  • kept me sane by outsourcing mundane data tasks

Changes in the Fed's decision-making

Posted in economics on March 31st, 2009 by Michael Ewens – Be the first to comment

Jim Hamilton is concerned that the Fed is changing the way it intervenes in the market and suggests the TALF may generate the false security of securitization that got us where we are now:

But the whole premise behind those Aaa ratings– that securitization could isolate a “safe” component of a pool of fundamentally risky loans– was deeply flawed. It is impossible to diversify away aggregate or systemic risk. All that the device did was to mislead investors into thinking they were protected from those nondiversifiable risks and push those risks onto the taxpayers and the Fed. Before we decide that securitization is the road out of our present difficulties, I would like a detailed and convincing explanation of why the past mistakes are not going to be repeated again.

Why not put GM through bankruptcy directly?

Posted in economics on March 30th, 2009 by Michael Ewens – Be the first to comment

And frankly, this plan doesn’t make survival look all that likely at anything remotely approaching GM’s current size–not if by “survival” we mean “weaning itself from taxpayer cash”.  The government can guarantee warrantees.  But it will be less effective at shedding all sorts of obligations than bankruptcy court.  The administration won’t be reorganizing the way a bankruptcy does; it will be negotiating.  To be sure, bankruptcy judges, too, negotiate, particularly among the various creditors.  But they do so with the power to cramdown firmly in their hands.  If the administration wants to wield that kind of power, it will have to find new and inventive threats to level at the creditors.  They may not find them.  And if they do, it’s not really in the best interests of the nation for the government to find lots of new and innovative ways to threaten private investors.

Megan McArdle on the  GM Bailout

The programmer behind mortgage bundling speaks

Posted in economics on March 30th, 2009 by Michael Ewens – Be the first to comment

I have been called the devil by strangers and “the Facilitator” by friends. It’s not uncommon for people, when I tell them what I used to do, to ask if I feel guilty. I do, somewhat, and it nags at me. When I put it out of mind, it inevitably resurfaces, like a shipwreck at low tide. It’s been eight years since I compiled a program, but the last one lived on, becoming the industry standard that seeded itself into every investment bank in the world.I wrote the software that turned mortgages into bonds. More here.

Newspaper Advertising Revenues Falling Across the Board

Posted in economics on March 28th, 2009 by Michael Ewens – Be the first to comment

Newspaper Advertising Revenues Falling Across the Board

New Research Suggests Index Funds > Mutual Funds

Posted in economics on March 26th, 2009 by Michael Ewens – Be the first to comment

In Fama and French’s new paper “Luck versus Skill in the Cross Section of Mutual Fund Alpha Estimates,” they find that mutual fund managers do not skills to earn expected returns above the market portfolio.  This result contrasts with the findings in Kosowski, Timmermann and White (2006) who find a set of mutual fund managers for 1975 – 2002 did outperform the market (not because of luck). Fama and French argue that these earlier results are suspect because the bootstraps ignored “common variation is fund returns.” Also, the different sample period and assumptions for sample inclusion may create data issues and survival bias. It will be interesting to see how this paper does in the next year or so.

Forecasting is Hard: Google can Make Money

Posted in economics on March 23rd, 2009 by Michael Ewens – Be the first to comment

But how will Google ever make money? There’s the rub. The company’s adamant refusal to use banner or other graphical ads eliminates what is the most lucrative income stream for rival search engines. Although Google does have other revenue sources, such as licensing and text-based advertisements, the privately held company’s business remains limited compared with its competitors’.

Northern Light (www.northernlight.com), which matches Google in many search-engine competitions, sells archived articles from hundreds of publications and builds intranet portals for companies. AltaVista (www.altavista.com), another competitor, has integrated its search engine into a broader portal strategy. “There isn’t really good evidence, frankly, that companies focused purely on search, as Google has been, can support themselves with that model,” says Northern Light Chief Technology Officer Marc Krellenstein.

Business magazines not so good at forecasting: Article from  December 7, 2000

S&P 500 Inflation-Adjusted Earnings Down 80%

Posted in economics on March 20th, 2009 by Michael Ewens – Be the first to comment

S&P 500 Inflation-Adjusted Earnings Down 80%