Today, Google announced two new services that are sure to be loved by data geeks. First is their BigQuery which lets you analyze “Terabytes of data, trillions of records.” This is great for people with large datasets. I wonder if a program like R(my favorite statistical analysis package) can read it? If so would R just pull down the data like it would from any other database? That would most likely result in a data.frame that is far too large for a standard computer to handle. Maybe R can be ran in a way that it hits the BigQuery service and leaves the data in there. Maybe even the processing can be done on Google’s end, allowing for much better computation time. This is something I’ve been dreaming of for a while now.
Jared Lander is the Chief Data Scientist of Lander Analytics a New York data science firm, Adjunct Professor at Columbia University, Organizer of the New York Open Statistical Programming meetup and the New York and Washington DC R Conferences and author of R for Everyone.