Is Upstart the right way to get college student start-ups funded?

In 2010, the movie “The Social Network” was released, which has had a tremendously positive effect on the computer science department at Yale; and from what I have heard, a similar effect has been observed across the country. My understanding (I have not seen this movie myself) is that the movie’s plot revolves around Mark Zuckerberg and his role in the formation of Facebook. In the time since the movie was released, the number of computer science majors at Yale has nearly quadrupled, and these majors  are increasingly looking at start-ups (either founding their own or joining existing ones) as options for when they graduate (instead of going to Wall Street and working as quants, which has historically been the popular career path for Yale CS majors).

In my opinion, this is unquestionably a good thing. Yale has some of the brightest minds of the next generation, and I feel a lot more confident about the future of our country when I see these great minds being applied to creating new entities and jobs and building something real, instead of being wasted in the zero-sum gain arms race of who can create the automatic trading algorithm that is epsilon better than anybody else’s.

One consequence of this start-up craze is that I get bombarded with requests from students who want to meet with me to discuss their start-up idea. This partly because I teach the “Intro to Programming” course at Yale which has had consistently between 120 and 150 students (many of whom are budding entrepreneurs) enrolled since the release of “The Social Network”, partly because the success of Hadapt is certainly no secret around Yale, and partly because I live on Yale’s campus and part of my job in this capacity is to serve as an adviser and mentor to undergraduates.

When I meet with these students I hear all kinds of ideas. Some of them are good, and some of them are bad. Some of them make me think about an area in a different way, and some of them are carbon copies of something that already exists. Some I could get excited about and some I couldn’t. But just about all of them have one thing in common: the students involved vastly overestimate their probability of success. I suppose this should not surprise me --- after all, these are Yale students that have been successful in just about everything they have ever done in their life. So it follows that they would expect their start-up to be successful. But even when I talk to students at other universities who have start-up ideas --- students who have not necessarily been so successful in their lives --- even they are totally convinced that their startup idea is unlikely to fail. It seems that there is a basic psychological flaw in the human mind --- we so desperately want our dreams to come true that we ignore statistical data about the probability of success and trick ourselves into believing that we are the statistical anomaly and will succeed where others have failed.

Many of these students find out the hard reality regarding their start-up idea when they attempt raise funding. They find out that investors are extremely conscious of the probability of success of a group of students with no experience, no reputation, and a limited network. Most of these start-ups fail to raise funding from professional investors. Some students give up at this point. Other students continue along with limited funding from friends and family in an attempt to create more meat around the bones of their idea and reduce risk for the professional investors. Most will eventually fail, while a rare few will succeed.

The outcome of all this is that despite all of these students eager to be entrepreneurs and start companies, very few student ideas receive funding, and most of these ideas never see the light of day. Whether or not this is a good thing is certainly up for debate, but my feeling is that it is a shame that so few student start-ups get funding.

Therefore, when I first heard of Upstart (I think it was in August), I was quite interested in the idea --- it proposed a way to get student start-ups funded. I signed up to receive e-mail updates, but did not hear from them for several months. However, on Monday of this week I received an update from them that they were open for business. I looked through the profiles of the students who were looking for funding and I saw that no fewer than 4 out of the (approximately) 20 profiles that were available online were from Yale University.

However, a deeper look at the Upstart Website reveals a problematic clause that is attached with the funding of the student start-up ideas. This is not a traditional crowdfunding model where investors receive equity in the start-up in exchange for their investment dollars. Instead, the investors get a percentage of the student’s income for a 10-year period in exchange for the investment. This way, in the likely event that the student’s start-up idea does not work out, the investor is able to receive a nice return on investment by taking a cut from the student’s hard earned salary when the student enters the workforce.

This does not seem right to me. On one side you have students who have an inaccurate view of the probability of success of their start-up, and on the other side you have investors who are looking to profit off of the boundless optimism and dreams of these students. These students, with no experience in the real world, no understanding of what skills are necessary to build a company, and a perception of entrepreneurship built more from Hollywood than the cold realities of business, are more than happy to mortgage a percentage of 10 years of future earnings for a chance to receive some short-term money about which they have no idea how to properly evaluate the costs vs. benefits.

In the traditional model, where the  investor receives equity in exchange for the investment, at least the investor is in the same boat as the student --- their interests are aligned and focused on making the start-up a success. With the Upstart model, you have almost the exact opposite. Since the salary of a founder is typically below-market in exchange for the equity the founder receives, the expected rate of return for the investor is actually higher if the student were to give up on the start-up and get a normal job. This is especially true when the investment rate of return for the investor is capped (as it is in Upstart), so that even if the start-up were to take off and the student were to become very wealthy from it, the return to the investor is not markedly different from what it would have been if the company had failed and the student later received a salary at market value. To exacerbate the situation, the investor-investee relationship in Upstart is supposed to be somewhat also a mentor-mentee relationship, which is particularly dangerous when interests are misaligned.

I think Upstart should be commended for trying to get more funding to college students with ideas for starting companies. And although I don’t know many people involved, the people I do know are good people and I highly doubt they are trying to do anything evil. (Jonathan Eng was a TA for my Introduction to Programming class for me 4 years ago, and he was a good and honest TA). However, I do not believe the people involved in Upstart realize how hard it is for students to accurately evaluate the costs and benefits of receiving funding in this way. Therefore I am highly concerned about this model as a way forward for student entrepreneurship.

IEEE Computer issue on the CAP Theorem

Due to Hurricane Sandy, Yale gave me a day off from teaching today and I have finally been able to get to a few things on my "to-do" list. One of them is to write a blog post about the IEEE Computer CAP Retrospective edition and make my paper that appeared inside of it publicly available.

Earlier this year, the IEEE Computer magazine came out with an issue largely devoted to a 12-year retrospective of the CAP theorem and contains several articles from distributed systems researchers that contribute various opinions and thoughts about CAP. The first article is from Eric Brewer, who coined the CAP theorem 12 years ago (though he points out in his article that it was actually 14 years ago). A PDF of Brewer’s article is available for free from: http://www.infoq.com/articles/cap-twelve-years-later-how-the-rules-have-changed. The second article is from Seth Gilbert and Nancy Lynch (the same Gilbert and Lynch that proved the CAP theorem 10 years ago). 

The third article is from me, and contains my criticisms of CAP that long-time readers of my blog will be familiar with. In particular, I point out that many people assume that modern NoSQL systems relax consistency guarantees in order to gain availability due to the constraints of the CAP theorem, when the reality is that these systems give up on consistency even in the absence of network partitions, which is not required according to the CAP theorem. The  reason why they give up on consistency is because of a desire to improve system latency, an increasingly important requirement in the modern impatient world. I then describe the latency-consistency tradeoff in more detail, and end the article with the PACELC reformulation of CAP that debuted on my blog over two years ago. With the permission of the IEEE, I am making a free version of this article available today. This article is the first time that the PACELC formulation and my thoughts on CAP appear in a scholarly article, which gives people a venue to refer to (bibtex code available here) when citing this work (you can stop citing a blog post!)

The fourth article is from Raghu Ramakrishnan, entitled “CAP and Cloud Data Management” and describes the PNUTS system that I have mentioned in the past as a good example of a system for which the consistency-latency tradeoff has had a more direct impact on the system design than the consistency-availability tradeoff of CAP. The fifth article is from Ken Birman, Daniel Freedman, Qi Huang, and Patrick Dowell of Cornell University on overcoming CAP with soft-state replication. Unfortunately, I cannot find a free link to Raghu’s article, but if you have an IEEE account, you can access it at at: http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6122007&tag=1. The Birman et. al. article can be found for free at: http://www.cs.cornell.edu/Projects/mrc/CAP.pdf.

If you have enjoyed my thoughts on CAP on this blog, I highly recommend you read each of these five articles. The Brewer article in particular acknowledges my past criticism of CAP not actually being about picking two of three out of C (consistency), A (availability), and P (partition tolerance) due to the fact that it does not make sense to reason about a system that is ‘CA’. (If there is no partition, any system can be both consistent and available --- the only question is what happens when there is a partition --- does consistency or availability get sacrificed?) Brewer uses this observation to lead into a nice generalization of consistency-availability tradeoff. In particular, when a partition occurs, the system does three things: (1) detect that the partition occurred, (2) enter a partition mode that may or may not limit some operations, and (3) initiate some sort of reconciliation algorithm when the partition is fixed. Depending on how these three things are implemented, it is  possible to obtain much of the spectrum between CP systems and AP systems. The article also contains a nice reference to the CRDT work by Shapiro et. al. at INRIA. Overall, I strongly support Brewer’s approach to navigating this tradeoff. It also fits nicely with Mehul Shah’s talk at HPTS in the way that the spectrum between consistency and availability is explicitly considered at system design time, rather than trying to bolt consistency on top of an AP (eventually consistent) system after the fact (a wildly suboptimal endeavor).

While most of Brewer’s article focused on the consistency-availability tradeoff, Brewer also briefly acknowledges that “in its classic interpretation, the CAP theorem ignores latency”, and that some systems reduce consistency for latency (he even refers to the PNUTS example I used in my original blog post). I remain convinced that PACELC is the best way to reason about both of these tradeoffs in a single formulation: if there is a partition (P) how does the system tradeoff between availability and consistency (A and C); else (E) when the system is running as normal in the absence of partitions, how does the system tradeoff between latency (L) and consistency (C)?

Defending Matt Welsh’s 'Startup University' Post

A week ago, Matt Welsh released a blog post on attaching a startup incubator to a university in order to create a funding model for some of the research that is performed at the university. Unfortunately, the beginning part of the blog post talked about the “inefficiency” of universities in terms of “producing real products” and the (perhaps overly dramatic) assertion that “nothing of practical value came out of [Matt’s] entire research career”. Although Matt has clarified that it was not his intention to indicate that the goal of academic research was to “produce real, shipping products that people could use”, many people interpreted the opening part of Matt’s post in that way, and reacted negatively (including, notably, Michael Mitzenmacher who responded in a comment and Joe Hellerstein who responded in his own blog post).

If we ignore the problems with the first part of Matt’s post, the rest of the post raises some important points and interesting ideas. As an academic who has spent large chunks of time spinning off a research project into a startup (HadoopDB was commercialized by Hadapt, which by most available metrics has been an example of a research lab-to-startup success story), many parts of Matt’s article rung true:

1. Matt’s statement: “Most universities make starting a company painfully difficult when it comes to questions of IP ownership [and] licensing” was certainly true for Hadapt. It took way too long, and way too much effort to get an agreement in place. Part of the problem was discussed in the comment thread of Matt’s post --- licensing patents are much better aligned with the core mission of a university than accepting equity in start-ups.
2.  Matt’s statement: “Most universities also make starting a company painfully difficult when it comes to […] forcing the academic's research to be dissociated with their commercial activities.” This was also true for me. I do not mean to criticize the university --- I absolutely understand the need for the conflict of interest safeguards because of the way that universities (and the assumptions of incoming students) are structured today. However, restructuring some of these assumptions in the way that Matt talks about may reduce the legal liabilities, and allow for fewer safeguards to have to be put in place. I also think that the students are hurt more than helped by some of these safeguards. For example, one of the PhD students involved in HadoopDB wanted to work part time for Hadapt while finishing his PhD. However, due to the COI legal complexities, he was forbidden from doing this and was forced to choose between Hadapt and the PhD program (he, of course, chose to take a leave of absence and join Hadapt).
3.  Matt’s statement that academics starting companies “involves a high degree of risk (potentially career-ending for pre-tenure faculty)” obviously resonates with me. Whether or not Hadapt is successful, it has certainly taken my time away from publishing papers (though obviously, I'm still trying to publish as much as I can --- see, for example, my last post on the Calvin project). Since publication quantity and quality remain key statistics for academic success, any conscious reduction of them comes with a clear risk.

The bottom line is that I absolutely agree with Matt’s assertion that there are a lot of extremely intelligent faculty in academic institutions across the world that have made the mental calculation and decided that the benefits do not outweigh the risks in spinning off a startup from an ongoing research project. Whether or not this is a bad thing is up for debate --- it is certainly not the core mission of a university to spin off companies or produce real-world products. However most universities do have some number of applied fields, and measuring impact in applied fields is often initiated by looking at real-world deployments of the research ideas. Starting companies is clearly the most direct mechanism for translating research ideas to real-world impact. Hence, it’s probably not a controversial statement to assert that reducing some of the barriers to starting companies would allow faculty in applied fields to increase their impact, the primary goal of research.

Therefore, allowing for explicit relationships between research groups and university-sponsored start-up incubators, where the university invests in a start-up, with proceeds from such investments being used to sponsor additional research in the department, is an idea worth considering. I would, however, change a few things about Matt’s proposal:

1.  I would not simply replace venture capital money with university money. Although it is easy to get into the trap of assuming that the venture capitalist simply trades investment dollars for equity in the company, it turns out that venture capitalists provide a lot of value in addition to their money. Seeing firsthand the difference at Hadapt before and after we got big-name venture capitalists behind us really drove this point home for me.  Therefore, I would recommend that the university partner with venture capitalists, or otherwise hire successful venture capitalists to work in-house (and continue to compensate them using the standard venture capital compensation schemes). Although the Kauffman report has recently shed some light into how poorly venture capital has performed over the last decade, the top venture capitalists have still done very well, and it is important to remember that the goal for the university is not to turn a profit on the investment, but rather to increase the number of startups coming out of the university, in order to increase the research impact. Break-even performance or even small amounts of losses are totally acceptable.
2.  The model will not work for any university. The location of the university is critical. Trying to get an incubator going for universities located in the middle of nowhere is a recipe for disaster. Technologists like to think that the technology that the company is commercializing is the most important factor in the company’s success. In fact, it falls way behind ‘market’ and ‘people’ as a determining factor. The company needs competent and experienced people throughout the organization --- the engineering team, marketing, sales, support, etc. Recruiting a competent team in a location where there have been small numbers of comparable companies is likely to be futile. Students from the university can only get you so far --- you need a mix of experienced people as well.
3.  There needs to be explicit mechanisms in place to reduce the risk for the faculty member. This means that the faculty member should get credit for certain company metrics at promotion or yearly evaluation time in addition to standard paper citation metrics. Company financial data is probably not a great metric, but customer counts of people actually using the technology, or even customer counts at “me-too” competitors could be used. Three years after publishing the original HadoopDB paper, there are real people using this technology to solve real problems. It’s pretty rare to see such an immediate impact, and it ought to count for something.

Obviously my own experiences have made me predisposed to liking Matt’s ideas, but I do encourage people to read the second half of his post independently of the first half.

If all these new DBMS technologies are so scalable, why are Oracle and DB2 still on top of TPC-C? A roadmap to end their dominance.

(This post is coauthored by Alexander Thomson and Daniel Abadi)
In the last decade, database technology has arguably progressed furthest along the scalability dimension. There have been hundreds of research papers, dozens of open-source projects, and numerous startups attempting to improve the scalability of database technology. Many of these new technologies have been extremely influential---some papers have earned thousands of citations, and some new systems have been deployed by thousands of enterprises.

So let’s ask a simple question: If all these new technologies are so scalable, why on earth are Oracle and DB2 still on top of the TPC-C standings? Go to the TPC-C Website with the top 10 results in raw transactions per second. As of today (May 16th, 2012), Oracle 11g is used for 3 of the results (including the top result), 10g is used for 2 of the results, and the rest of the top 10 is filled with various versions of DB2. How is technology designed decades ago still dominating TPC-C? What happened to all these new technologies with all these scalability claims?

The surprising truth is that these new DBMS technologies are not listed in the TPC-C top ten results not because that they do not care enough to enter, but rather because they would not win if they did.

To understand why this is the case, one must understand that scalability does not come for free. Something must be sacrificed to achieve high scalability. Today, there are three major categories of tradeoff that can be exploited to make a system scale. The new technologies basically fall into two of these categories; Oracle and DB2 fall into a third. And the later parts of this blog post describes research from our group at Yale that introduces a fourth category of tradeoff that provides a roadmap to end the dominance of Oracle and DB2.

These categories are:

(1) Sacrifice ACID for scalability. Our previous post on this topic discussed this in detail. Basically we argue that a major class of new scalable technologies fall under the category of “NoSQL” which achieves scalability by dropping ACID guarantees, thereby allowing them to eschew two phase locking, two phase commit, and other impediments to concurrency and processor independence that hurt scalability. All of these systems that relax ACID are immediately ineligible to enter the TPC-C competition since ACID guarantees are one of TPC-C’s requirements. That’s why you don’t see NoSQL databases in the TPC-C top 10---they are immediately disqualified.

(2) Reduce transaction flexibility for scalability. There are many so-called “NewSQL” databases that claim to be both ACID-compliant and scalable. And these claims are true---to a degree. However, the fine print is that they are only linearly scalable when transactions can be completely isolated to a single “partition” or “shard” of data. While these NewSQL databases often hide the complexity of sharding from the application developer, they still rely on the shards to be fairly independent. As soon as a transaction needs to span multiple shards (e.g., update two different user records on two different shards in the same atomic transaction), then these NewSQL systems all run into problems. Some simply reject such transactions. Others allow them, but need to perform two phase commit or other agreement protocols in order to ensure ACID compliance (since each shard may fail independently). Unfortunately, agreement protocols such as two phase commit come at a great scalability cost (see our 2010 paper that explains why). Therefore, NewSQL databases only scale well if multi-shard transactions (also called “distributed transactions” or “multi-partition transactions”) are very rare. Unfortunately for these databases, TPC-C models a fairly reasonable retail application where customers buy products and the inventory needs to be updated in the same atomic transaction. 10% of TPC-C New Order transactions involve customers buying products from a “remote” warehouse, which is generally stored in a separate shard. Therefore, even for basic applications like TPC-C, NewSQL databases lose their scalability advantages. That’s why the NewSQL databases do not enter TPC-C results --- even just 10% of multi-shard transactions causes their performance to degrade rapidly.

(3) Trade cost for scalability. If you use high end hardware, it is possible to get stunningly high transactional throughput using old database technologies that don’t have shared-nothing horizontally scalability. Oracle tops TPC-C with an incredibly high throughput of 500,000 transactions per second. There exists no application in the modern world that produces more than 500,000 transactions per second (as long as humans are initiating the transactions---machine-generated transactions are a different story). Therefore, Oracle basically has all the scalability that is needed for human scale applications. The only downside is cost---the Oracle system that is able to achieve 500,000 transactions per second costs a prohibitive $30,000,000!

Since the first two types of tradeoffs are immediate disqualifiers for TPC-C, the only remaining thing to give up is cost-for-scale, and that’s why the old database technologies are still dominating TPC-C. None of these new technologies can handle both ACID and 10% remote transactions.

A fourth approach...

TPC-C is a very reasonable application. New technologies should be able to handle it. Therefore, at Yale we set out to find a new dimension in this tradeoff space that could allow a system to handle TPC-C at scale without costing $30,000,000. Indeed, we are presenting a paper next week at SIGMOD (see the full paper) that describes a system that can achieve 500,000 ACID-compliant TPC-C New Order transactions per second using commodity hardware in the cloud. The cost to us to run these experiments was less than $300 (of course, this is renting hardware rather than buying, so it’s hard to compare prices --- but still --- a factor of 100,000 less than $30,000,000 is quite large).

Calvin, our prototype system designed and built by a large team of researchers at Yale that include Thaddeus Diamond, Shu-Chun Weng, Kun Ren, Philip Shao, Anton Petrov, Michael Giuffrida, and Aaron Segal (in addition to the authors of this blog post), explores a tradeoff very different from the three described above. Calvin requires all transactions to be executed fully server-side and sacrifices the freedom to non-deterministically abort or reorder transactions on-the-fly during execution. In return, Calvin gets scalability, ACID-compliance, and extremely low-overhead multi-shard transactions over a shared-nothing architecture. In other words, Calvin is designed to handle high-volume OLTP throughput on sharded databases on cheap, commodity hardware stored locally or in the cloud. Calvin significantly improves the scalability over our previous approach to achieving determinism in database systems.

Scaling ACID

The key to Calvin’s strong performance is that it reorganizes the transaction execution pipeline normally used in DBMSs according to the principle: do all the "hard" work before acquiring locks and beginning execution. In particular, Calvin moves the following stages to the front of the pipeline:

• Replication. In traditional systems, replicas agree on each modification to database state only after some transaction has made the change at some "master" replica. In Calvin, all replicas agree in advance on the sequence of transactions that they will (deterministically) attempt to execute.
• Agreement between participants in distributed transactions. Database systems traditionally use two-phase commit (2PC) to handle distributed transactions. In Calvin, every node sees the same global sequence of transaction requests, and is able to use this already-agreed-upon information in place of a commit protocol.
• Disk accesses. In our VLDB 2010 paper, we observed that deterministic systems performed terribly in disk-based environments due to holding locks for the 10ms+ duration of reading the needed data from disk, since they cannot reorder conflicting transactions on the fly. Calvin gets around this setback by prefetching into memory all records that a transaction will need during the replication phase---before locks are even acquired.

As a result, each transaction’s user-specified logic can be executed at each shard with an absolute minimum of runtime synchronization between shards or replicas to slow it down, even if the transaction’s logic requires it to access records at multiple shards. By minimizing the time that locks are held, concurrency can be greatly increased, thereby leading to near-linear scalability on a commodity cluster of machines.

Strongly consistent global replication

Calvin’s deterministic execution semantics provide an additional benefit: replicating transactional input is sufficient to achieve strongly consistent replication. Since replicating batches of transaction requests is extremely inexpensive and happens before the transactions acquire locks and begin executing, Calvin’s transactional throughput capacity does not depend at all on its replication configuration.

In other words, not only can Calvin can run 500,000 transactions per second on 100 EC2 instances in Amazon’s US East (Virginia) data center, it can maintain strongly-consistent, up-to-date 100-node replicas in Amazon’s Europe (Ireland) and US West (California) data centers---at no cost to throughput.

Calvin accomplishes this by having replicas perform the actual processing of transactions completely independently of one another, maintaining strong consistency without having to constantly synchronize transaction results between replicas. (Calvin’s end-to-end transaction latency does depend on message delays between replicas, of course---there is no getting around the speed of light.)

Flexible data model

So where does Calvin fall in the OldSQL/NewSQL/NoSQL trichotomy?

Actually, nowhere. Calvin is not a database system itself, but rather a transaction scheduling and replication coordination service. We designed the system to integrate with any data storage layer, relational or otherwise. Calvin allows user transaction code to access the data layer freely, using any data access language or interface supported by the underlying storage engine (so long as Calvin can observe which records user transactions access). The experiments presented in the paper use a custom key-value store. More recently, we’ve hooked Calvin up to Google’s LevelDB and added support for SQL-based data access within transactions, building relational tables on top of LevelDB’s efficient sorted-string storage.

From an application developer’s point of view, Calvin’s primary limitation compared to other systems is that transactions must be executed entirely server-side. Calvin has to know in advance what code will be executed for a given transaction. Users may pre-define transactions directly in C++, or submit arbitrary Python code snippets on-the-fly to be parsed and executed as transactions.

For some applications, this requirement of completely server-side transactions might be a difficult limitation. However, many applications prefer to execute transaction code on the database server anyway (in the form of stored procedures), in order to avoid multiple round trip messages between the database server and application server in the middle of a transaction.

If this limitation is acceptable, Calvin presents a nice alternative in the tradeoff space to achieving high scalability without sacrificing ACID or multi-shard transactions. Hence, we believe that our SIGMOD paper may present a roadmap for overcoming the scalability dominance of the decades-old database solutions on traditional OLTP workloads. We look forward to debating the merits of this approach in the weeks ahead (and Alex will be presenting the paper at SIGMOD next week).