Natural Language in Voice Search

Posted by Jakob Uszkoreit, Software Engineer

On July 26 and 27, we held our eighth annual Computer Science Faculty Summit on our Mountain View Campus. During the event, we brought you a series of blog posts dedicated to sharing the Summit's talks, panels and sessions, and we continue with this glimpse into natural language in voice search. --Ed

At this year’s Faculty Summit, I had the opportunity to showcase the newest version of Google Voice Search. This version hints at how Google Search, in particular on mobile devices and by voice, will become increasingly capable of responding to natural language queries.

I first outlined the trajectory of Google Voice Search, which was initially released in 2007. Voice actions, launched in 2010 for Android devices, made it possible to control your device by speaking to it. For example, if you wanted to set your device alarm for 10:00 AM, you could say “set alarm for 10:00 AM. Label: meeting on voice actions.” To indicate the subject of the alarm, a meeting about voice actions, you would have to use the keyword “label”! Certainly not everyone would think to frame the requested action this way. What if you could speak to your device in a more natural way and have it understand you?

At last month’s Google I/O 2012, we announced a version of voice actions that supports much more natural commands. For instance, your device will now set an alarm if you say “my meeting is at 10:00 AM, remind me”. This makes even previously existing functionality, such as sending a text message or calling someone, more discoverable on the device -- that is, if you express a voice command in whatever way feels natural to you, whether it be “let David know I’ll be late via text” or “make sure I buy milk by 3 pm”, there is now a good chance that your device will respond how you anticipated it to.

I then discussed some of the possibly unexpected decisions we made when designing the system we now use for interpreting natural language queries or requests. For example, as you would expect from Google, our approach to interpreting natural language queries is data-driven and relies heavily on machine learning. In complex machine learning systems, however, it is often difficult to figure out the underlying cause for an error: after supplying them with training and test data, you merely obtain a set of metrics that hopefully give a reasonable indication about the system’s quality but they fail to provide an explanation for why a certain input lead to a given, possibly wrong output.

As a result, even understanding why some mistakes were made requires experts in the field and detailed analysis, rendering it nearly impossible to harness non-experts in analyzing and improving such systems. To avoid this, we aim to make every partial decision of the system as interpretable as possible. In many cases, any random speaker of English could look at its possibly erroneous behavior in response to some input and quickly identify the underlying issue - and in some cases even fix it!

We are especially interested in working with our academic colleagues on some of the many fascinating research and engineering challenges in building large-scale, yet interpretable natural language understanding systems and devising the machine learning algorithms this requires.

Read More

New Challenges in Computer Science Research

Posted by Jeff Walz, Head of University Relations

Yesterday afternoon at the 2012 Computer Science Faculty Summit, there was a round of lightning talks addressing some of the research problems faced by Google across several domains. The talks pointed out some of the biggest challenges emerging from increasing digital interaction, which is this year’s Faculty Summit theme.

Research Scientist Vivek Kwatra kicked things off with a talk about video stabilization on YouTube. The popularity of mobile devices with cameras has led to an explosion in the amount of video people capture, which can often be shaky. Vivek and his team have found algorithmic approaches to make casual videos look more professional by simulating professional camera moves. Their stabilization technology vastly improves the quality of amateur footage.

Next, Ed Chi (Research Scientist) talked about social media focusing on the experimental circle model that characterizes Google+. Ed is particularly interested in how social interaction on the web can be designed to mimic live communication. Circles on Google+ allow a user to manage their audience and share content in a targeted fashion, which reflects face-to-face interaction. Ed discussed how, from an HCI perspective, the challenge going forward is the need to consider the trinity of social media: context, audience, content.

John Wilkes, Principal Software Engineer, talked about cluster management at Google and the challenges of building a new cluster manager-- that is, an operating system for a fleet of machines. Everything at Google is big and a consequence of operating at such tremendous scale is that machines are bound to fail. John’s team is working to make things easier for internal users enabling our ability to respond to more system requests. There are several hard problems in this domain, such as issues with configuration, making it as easy as possible to run a binary, increasing failure tolerance, and helping internal users understand their own needs as well as the behavior and performance of their system in our complicated distributed environment.

Research Scientist and coffee connoisseur Alon Halevy took to the podium to confirm that he did indeed author an empirical book on coffee, and also talked with attendees about structured data on the web. Structured data is comprised of hundreds of millions of (relatively small) tables of data, and Alon’s work is focused on enabling data enthusiasts to discover and visualize those data sets. Great possibilities open up when people start combining data sets in meaningful ways, which inspired the creation of Fusion Tables. An example is a map made in the aftermath of the 2011 earthquake and tsunami in Japan, that shows natural disaster data alongside the locations of the world’s nuclear plants. Moving forward, Alon’s team will continue to think about interesting things that can be done with data, and the techniques needed to distinguish good data from bad data.

To wrap up the session, Praveen Paritosh did a brief, but deep dive into the Knowledge Graph, an intelligent model that understands real-world entities and their relationships to one another-- things, not strings-- which launched earlier this year.

The Google Faculty Summit continued today with more talks, and breakout sessions centered on our theme of digital interaction. Check back for additional blog posts in the coming days.

Read More

Education in the Cloud

Posted by Andrea Held, University Relations

In the last 10 years, we’ve seen a major transition from stand-alone applications that run on desktop computers to applications running in the cloud. Unfortunately, many computer science students don’t have the opportunity to learn and work in the cloud due to a lack of resources in traditional undergrad programs. Without this access students are limited to the resources their school can provide.

So today, we’re announcing a new award program: the Google App Engine Education Awards. We are excited because Google App Engine can teach students how to build sophisticated large-scale systems in the cloud without needing access to a large physical network.

Google App Engine can be used to build mobile or social applications, traditional browser-based applications, or stand-alone web services that scale to millions of users with ease. The Google App Engine infrastructure and storage tools are useful for collecting and analyzing educational data, building a learning management system to organize courses, or implementing a teacher forum for exchanging ideas and practices. All of these adaptations of the Google App Engine platform will use the same infrastructure that powers Google.

We invite teachers at universities across the United States to submit a proposal describing how to use Google App Engine for their course development, educational research or tools, or for student projects. Selected proposals will receive $1,000 in App Engine credits.

If you teach at an accredited college, university or community college in the US, we encourage you to apply. You can submit a proposal by filling out this form. The application deadline is midnight PST August 31, 2012.

Read More

Big Pictures with Big Messages

Posted by Maggie Johnson, Director of Education and University Relations

Google’s Eighth Annual Computer Science Faculty Summit opened today in Mountain View with a fascinating talk by Fernanda Viégas and Martin Wattenberg, leaders of the data visualization group at our Cambridge office. They provided insight into their design process in visualizing big data, by highlighting Google+ Ripples and a map of the wind they created.

To preface his explanation of the design process, Martin shared that his team “wants visualization to be ‘G-rated,’ showing the full detail of the data - there’s no need to simplify it, if complexity is done right.” Martin discussed how their wind map started as a personal art project, but has gained interest particularly among groups that are interested in information on the wind (sailors, surfers, firefighters). The map displays surface wind data from the US National Digital Forecast Database and updates hourly. You can zoom around the United States looking for where the winds are fastest - often around lakes or just offshore - or check out the gallery to see snapshots of the wind from days past.

Fernanda discussed the development of Google+ Ripples, a visualization that shows how news spreads on Google+. The visualization shows spheres of influence and different patterns of spread. For example, someone might post a video to their Google+ page and if it goes viral, we’ll see several circles in the visualization. This depicts the influence of different individuals sharing content, both in terms of the number of their followers and the re-shares of the video, and has revealed that individuals are at times more influential than organizations in the social media domain.

Martin and Fernanda closed with two important lessons in data visualization: first, don’t “dumb down” the data. If complexity is handled correctly and in interesting ways, our users find the details appealing and find their own ways to interact with and expand upon the data. Second, users like to see their personal world in a visualization. Being able to see the spread of a Google+ post, or zoom in to see the wind around one’s town is what makes a visualization personal and compelling-- we call this the “I can see my house from here” feature.

The Faculty Summit will continue through Friday, July 27 with talks by Googlers and faculty guests as well as breakout sessions on specific topics related to this year’s theme of digital interactions. We will be looking closely at how computation and bits have permeated our everyday experiences via smart phones, wearable computing, social interactions, and education.

We will be posting here throughout the summit with updates and news as it happens.

Read More

Site Reliability Engineers: “solving the most interesting problems”

Posted by Chris Reid, Sydney Staffing team

I recently sat down with Ben Appleton, a Senior Staff Software Engineer, to talk about his recent move from Software Engineer (SWE) on the Maps team to Site Reliability Engineering (SRE). In the interview, Ben explains why he transitioned from a pure development role to a role in production, and how his work has changed:

Chris: Tell us about your path to Google.
Ben: Before I joined Google I didn’t consider myself a “software engineer”. I went to the University of Queensland and graduated with a Bachelor’s Degree in Electrical Engineering and Mathematics, before going on to complete a Ph.D. My field of research was image segmentation, extending graph cuts to continuous space for analyzing X-rays and MRIs. At a conference in France I met a friend of my Ph.D. advisor’s, and he raved about Google, commenting that they were one of the only companies that really understood technology. I’d already decided academia wasn’t for me, so I interviewed for a general Software Engineering role at Google. I enjoyed the interviews, met some really smart people, and learned about some interesting stuff they were working on. I joined the Maps team in Sydney in 2005 and spent the next 6 years working on the Maps API.

Chris: Tell us about some of the coolest work you did for Google Maps, and how you applied your research background.
Ben: My background in algorithms and computational geometry was really useful. We were basically making browsers do stuff they’re not designed to do, such as rendering millions of vectors or warping images, inventing techniques as we went. On the server-side we focused on content distribution, pushing tiles or vectors from Google servers down through caches to the user’s browser, optimizing for load and latency at every stage. On the client-side, we had to make the most of limited processors with new geometric algorithms and clever prefetching to hide network latency. It was really interesting work.

Chris: I understand you received company-wide recognition when you were managing the Maps API team. Tell us more about what that entailed.
Ben: In September 2008, when managing the Maps API, my team received an award that was recognized Google-wide, which is a big honor. My main contributions were latency optimizations, stability, enterprise support, and Street View integration. The award was in recognition of strong sustained growth of the Maps API, in relation to the number of sites using it, and total views per day. Currently the Google Maps API is serving more than 600,000 websites.

Chris: So what prompted the move to Site Reliability Engineering (SRE)?
Ben: In my experience, a lot of software engineers don’t understand what SREs do. I’d worked closely with SREs, particularly those in Sydney supporting Maps, and had formed a high opinion of them. They’re a very strong team - they’re smart and they get things done. After 6 years working on the Maps API I felt it was time for a change. In Sydney there are SWE teams covering most of the product areas, including Chrome and Apps, Social and Blogger, Infrastructure Networking and the Go programming language, as well as Maps and GeoCommerce. I talked to all of them, but chose SRE because in my opinion, they’re solving the most interesting problems.

Chris: How would you describe SRE?
Ben: It really depends on the individual. At one end are the Systems Administrator types, sustaining ridiculously large systems. But at the other end are the Software Engineers like me. As SREs get more experienced this distinction tends to be blurred. The best SREs think programmatically even if they don’t do the programming. For me, I don’t see a difference in my day-to-day role. When I was working on the Maps API I was the primary on-call one week in three, whereas in SRE the typical on-call roster is one week in six. When you’re primary on-call it just means you’re the go-to person for the team, responsible for when something breaks or pushing new code into production. I was spending 50% of my time doing coding and development work, and as an SRE this has increased to 80%.

Chris: Wow! So as an SRE in Production, you’re spending less time on-call and more time writing code than you were as a SWE on the Maps team?
Ben: Yes! I’m not managing a team now, but I’m definitely spending more time coding than I was before. I guess the average SRE spends 50% of their time doing development work, but as I said, it depends on the person and it ranges from 20-80%.

Chris: What does your team do?
Ben: In Sydney there are SRE teams supporting Maps, Blogger, App Engine, as well as various parts of the infrastructure and storage systems. I’m working on Blobstore, an infrastructure storage service based on Bigtable which simplifies building and deploying applications that store users' binary data (BLOBs, or "Binary Large OBjects"). Example BLOBs include images, videos, or email attachments - any data objects that are immutable and long-lived. The fact that we're storing user data means that Blobstore must be highly available for reads and writes, be extremely reliable (so that we never lose data), and be efficient in terms of storage usage (so that we can provide large amounts of storage to users at low cost).

Chris: Tell us more about some of the problems you’re solving, and how they differ with those you faced as a SWE in a development role.
Ben: With the massive expansion in online data storage, we’re solving problems at a scale never before seen. Due to the global nature of our infrastructure, we think in terms of load balancing at many levels: across regions, across data centers within a region, and across machines within a data center. The problems we’re facing in SRE are much closer to the metal. We’re constantly optimizing resource allocation and efficiency and scalability of Google’s massive computer systems, as opposed to developing new features for a product like Maps. So the nature of the work is very similar to SWE, but the problems are bigger and there is a strong focus on scale.

Chris: Are you planning on staying in SRE for a while?
Ben: Yeah. I signed up for a six month rotation program called “Mission Control,” the goal of which is to teach engineers to understand the challenges of building and operating a high reliability service at Google scale. In other words, it’s an SRE training program. In my first three months of Mission Control I’ve been on-call twice, and always during office hours so there were SREs to help me when I got into trouble...which I did. I’ve got no intention of going back to SWE at the end of the six months and plan to stay in SRE for at least a few years. Right now the problems seem more interesting. For example, last year’s storage solutions are facing additional strain from the growth of Gmail, Google+ and Google Drive. So you’re constantly reinventing.

Chris: What advice do you have for Software Engineers contemplating a role in SRE?
Ben: SRE gives you the opportunity to work on infrastructure at a really big scale in a way you don’t get to in SWE. Whereas SWE is more about developing new features, SRE is dealing with bigger problems and more complex engineering due to the sheer scale. SRE is a great way to learn how systems really work in order to become a great engineer.

If you’re interesting in applying for a Site Reliability Engineering role, please note that we advertise the roles in several different ways to reflect the diversity of the team. The two main roles are “Software Engineer, Google.com” and “Systems Engineer, Google.com”. We use the term “Google.com” to signify that the roles are in Production as opposed to R&D. You can find all the openings listed on the Google jobs site. We’re currently hiring across many regions, including Sydney in Australia, and of course Mountain View in California.

Read More

Google at SIGMOD/PODS 2012

Posted by Anish Das Sarma, Research Scientist and Jeff Shute, Software Engineer

Over the years, SIGMOD has expanded beyond a traditional "database" conference to include several areas related to information management. This year’s ACM SIGMOD/PODS conference (on Management of Data, and Principles of Database Systems), held in Scottsdale, Arizona was no different. We were impressed by the wide variety of researchers from industry and academia alike the conference attracted, and enjoyed learning how others are pushing the limits of scalability in data storage and processing. In addition to an excellent set of papers on a large number of topics, we saw a couple of recurring themes:

1) Data Visualization

• Pat Hanrahan from Stanford gave a keynote on some of the challenges involved in building systems to enable "data enthusiasts" to manage and visualize data. 

• Google’s Fusion Tables group also had a paper on this topic: Efficient Spatial Sampling of Large Geographical Tables, by Anish Das Sarma, Hongrae Lee, Hector Gonzalez, Jayant Madhavan, Alon Halevy. (This paper has been invited to a TODS special issue on best papers of SIGMOD 2012). 

• A similar effort from the University of Washington was presented as a demo: VizDeck: Self-Organizing Dashboards for Visual Analytics, by Alicia Key, Bill Howe, Daniel Perry, Cecilia Aragon.

2) Big Data

As has been the case for the last couple of years, “Big Data" has been of ever-growing interest to the entire community, particularly from industry. Google presented a talk on F1, a new distributed database system we’ve built to power the AdWords system. A complex business application like AdWords has different requirements than many systems at Google that often use storage systems like Bigtable. We have a single database shared by hundreds of developers and systems, so we need the robustness and ease of use we’re used to from traditional databases. F1 is built to scale like Bigtable, without giving up the database features we also need, like strong consistency, ACID transactions, schema enforcement, and most importantly, SQL query.

There’s been a widespread trend over the last several years away from databases, towards highly scalable “NoSQL” systems. We don’t think that trade-off is necessary, and were happy to see several other speakers advocate a similar theme -- yes, databases are useful, and developers shouldn’t need to give up database features and ease of use in the name of scalability.

This theme was supported by an industry session on Big Data featuring talks from other companies: Facebook (TAO: How Facebook Serves the Social Graph), Twitter (Large-Scale Machine Learning at Twitter), and Microsoft (Recurring Job Optimization in Scope). Googler Kirsten LeFevre was a panelist on the "Perspectives on Big Data" panel organized by Surajit Chaudhuri from Microsoft, and also featuring Donald Kossmann from ETHZ, Sam Madden from MIT, and Anand Rajaraman from Walmart Labs. Last but not the least, Surajit Chaudhuri also gave an excellent keynote outlining some of the research challenges that the new era of "Big Data and Cloud" poses.

As has been the practice for several years now, to continue generating great interest in data management research, SIGMOD has been organizing panels such as this year's "New Research Symposium" (which included Anish Das Sarma from Google as a panelist).

In addition to sponsoring the conference, many Googlers attended contributing to a robust presence and affording us the opportunity to interact with the broader information management community. We've been pushing the frontiers of science with cutting-edge research in many aspects of data management, and we were eager to share our innovations and see what others have been working on. We found Amin Vahdat's keynote on the intersection of Networking and Databases to be a highlight of Google’s participation, which also included presenting papers, participating on panels, and taking part in planning and program committees:

Program Committee Members

Anish Das Sarma, Venkatesh Ganti, Zoltan Gyongyi, Alon Halevy (Tutorials Chair), Kristen LeFevre, Cong Yu

Talks

Symbiosis in Scale Out Networking and Data Management
Amin Vahdat, Google (Keynote)

F1-The Fault-Tolerant Distributed RDBMS Supporting Google's Ad Business
Jeff Shute, Mircea Oancea, Stephan Ellner, Ben Handy, Eric Rollins, Bart Samwel, Radek Vingralek, Chad Whipkey, Xin Chen, Beat Jegerlehner, Kyle Littlefield, Phoenix Tong (Googlers)

Finding Related Tables
Anish Das Sarma, Lujun Fang, Nitin Gupta, Alon Halevy, Hongrae Lee, Fei Wu, Reynold Xin, Cong Yu (Googlers)

Papers

CloudRAMSort: Fast and Efficient Large-Scale Distributed RAM Sort on Shared-Nothing Cluster
Changkyu Kim, Jongsoo Park, Nadathur Satish, Hongrae Lee (Google), Pradeep Dubey, Jatin Chhugani

Efficient Spatial Sampling of Large Geographical Tables
Anish Das Sarma, Hongrae Lee, Hector Gonzalez, Jayant Madhavan, Alon Halevy (Googlers)

Panels

Perspectives on Big Data Plenary Session: Privacy and Big Data 
Kristen LeFevre, Google

SIGMOD New Researcher Symposium - How to be a good advisor/advisee? 
Anish Das Sarma, Google

Overall, this year’s SIGMOD was a great conference, widely attended by researchers from industry and academia, and comprised of a very interesting mix of research presentations and discussions. Google had a good showing at the conference, and we look forward to continuing this trend in the coming years.

Read More

Reflections on the Google Faculty Institute

Posted by Maggie Johnson, Director of Education and University Relations

Extending the school year one day can result in a year’s worth of learning. This proved true on June 8, when the 2011 Google Faculty Institute (GFI) cohort were welcomed back for a day, to share best practices and perspectives from their funded research over the 2011-12 school year.

For the past year, the GFI Fellows collaborated across 16 California State University campuses, Stanford and UC Berkeley, to execute on ten research initiatives proposed on the final day of the conference. GFI themes of faculty collaboration, project-based learning, universal design and others were implemented in the Fellows’ projects, each of which focused on ways to enhance teaching practices through the use of educational technologies.

At the GFI Redux earlier this month, participants reviewed research initiatives, attended panel discussions, and defined plans for the 2012-13 school year. In a packed day of sessions, the cohort showcased projects ranging from mobile application development to geospatial tool utilization to the success of the flipped classroom. Some highlights of GFI projects:

• Making Teachers “Appy” presented workshops on UC and CSU campuses on mobile application development using App Inventor. While building confidence with new technologies, participants learned to create their own applications to enhance classroom instruction.

• Bird’s Eye Detective encouraged CSU pre-service teachers to explore the world from a new perspective utilizing geospatial tools including Google Earth, Google Maps, and Fusion Tables.

• Transforming STEM Educators included nine hands-on workshops on three CSU campuses, presenting creative ways to engage students in science and engineering courses through the use of technology.

• CSU Digital Learning Ambassadors are faculty creating collaborative communities and customized initiatives from the inside. Initiatives include tech infusion prizes, Hangouts on Air for academic discussions, and webinars.

The Google Faculty Institute served as a catalyst and incubator for innovative educational technology. Congratulations to the GFI Fellows on a year of excellent research and application.

Read More

Google Research Awards: Summer, 2012

Posted by Maggie Johnson, Director of Education, University Relations

We’ve just finished the review process for the latest round of the Google Research Awards, which is our bi-annual open call for proposals on research in areas of mutual interest with Google. Our funding provides full-time faculty the opportunity to fund a graduate student and work directly with Google research scientists and engineers.

This round, we are funding 104 awards across 21 different focus areas for a total of nearly $6 million. The subject areas that received the highest level of support this time were systems and infrastructure, human computer interaction, and mobile. In addition, 28% of the funding was awarded to universities outside the U.S.

Given that our program is merit-based, we make funding decisions via committees of experts, who assess each proposal by its impact, innovation, relevance to Google, and other factors. Over the past two years, we have seen significant growth in the Research Award program. This round, we had 815 proposals—up 11% from last round, which required 1,946 reviews by 654 reviewers.

Our award committees represent a microcosm of Research @ Google. Not only do we work with research scientists in making funding decisions, but also engineers—many of whom have advanced degrees in Computer Science. Our research organization has a similar make-up: both research scientists and engineers working together on innovative projects that are product-focused and relevant to our customers.

Congratulations to the well-deserving recipients of this round’s awards. If you are interested in applying for the next round (deadline is October 15), please visit our website for more information.

Read More

Our Unique Approach to Research

Posted by Alfred Spector, Vice President of Research and Special Initiatives

Google started as a research project—and research has remained a core part of our culture. But we also do research differently than many other places. To shed more light on Google’s unique approach to research, Peter Norvig (Director of Research), Slav Petrov (Senior Research Scientist) and I recently published a paper, “Google’s Hybrid Approach to Research,” in the July issue of Communications of the ACM.

   

In the paper, we describe our hybrid approach to research, which integrates research and development to maximize our impact on users and the speed at which we make progress. Our model allows us to work at unparalleled scale and conduct research in vivo on real systems with millions of users, rather than on artificial prototypes. This yields not only innovative research results and new technologies, but valuable new capabilities for the company—think of MapReduce, Voice Search or open source projects such as Android and Chrome. 

Breaking up long-term research projects into shorter-term, measurable components is another aspect of our integrated model. This is not to say our model precludes longer-term objectives, but we try to achieve these in stages. For example, Google Translate is a multi-year project characterized by the need for both research and complex systems, but we’ve achieved many small objectives along the way—such as adding languages over time for a current total of 64, developing features like two-step translation functionality, enabling users to make corrections, and consideration of syntactic structure.

Overall, our success in the areas of systems, speech recognition, language translation, machine learning, market algorithms, computer vision and many other areas has stemmed from our hybrid research approach. While there are risks associated with the close integration of research and development activities—namely the concern that research will take a back seat in favor of shorter-term projects—we mitigate those by focusing on the user and empirical data, maintaining a flexible organizational structure, and engaging with the academic community. We have a portfolio of timescales, with some researchers working with engineers to rapidly iterate on existing products, and others working on forward-looking projects that will benefit people in the future.

We hope “Google’s Hybrid Approach to Research” helps explain our method. We feel it will bring some clarification and transparency to our approach, and perhaps merit consideration by other technology companies and academic labs that organize research differently.

To learn more about what we do and see see real-time applications of our hybrid research model, add Research at Google to your circles on Google+.

(Cross-posted on the Official Google Blog) 

Read More