The War in the Clouds : How Massive Data Centers will Change the War Venue・雲の上の戦争 : 巨大データセンターがデジタルコンテンツの新たな戦場になる

I have been writ­ing this arti­cle since March 17th, found it in my “not pub­lished” tray so I decided to fin­ish it today, There are dat­a­cen­ters and DATACENTES.   What is a dat­a­cen­ter ? Data cen­ter infra­struc­ture lay­ers are power, cool­ing, tele­com, data rooms and net­work oper­a­tions cen­ter. In May 2008, Jeff Dean spoke at the Google I/O con­fer­ence high­light­ing some infor­ma­tion on the inner work­ings of their dat­a­cen­ter and ambi­tious plans. With already 36 dat­a­cen­ter around the world in 2008 with over 200,000 servers, that is a lot.

Giz­modo

Here are some infor­ma­tion from Wiki on Google’s datacenter

Orig­i­nal hardware

The orig­i­nal hard­ware (circa 1998) that was used by Google when it was located at Stan­ford Uni­ver­sity included:^[7]

• Sun Ultra II with dual 200 MHz proces­sors, and 256 MB of RAM. This was the main machine for the orig­i­nal Back­rub system.
• 2 × 300 MHz Dual Pen­tium II Servers donated by Intel, they included 512 MB of RAM and 9 × 9 GB hard dri­ves between the two. It was on these that the main search ran.
• F50 IBM RS/6000 donated by IBM, included 4 proces­sors, 512 MB of mem­ory and 8 × 9 GB hard drives.
• Two addi­tional boxes included 3 × 9 GB hard dri­ves and 6 x 4 GB hard dri­ves respec­tively (the orig­i­nal stor­age for Back­rub). These were attached to the Sun Ultra II.
• IBM disk expan­sion box with another 8 × 9 GB hard dri­ves donated by IBM.
• Home­made disk box which con­tained 10 × 9 GB SCSI hard drives.

Cur­rent hardware

Servers are commodity-class x86 PCs run­ning cus­tomized ver­sions of Linux. The goal is to pur­chase CPU gen­er­a­tions that offer the best per­for­mance per dol­lar, not absolute per­for­mance.^[8] Esti­mates of the power required for over 450,000 servers range upwards of 20 megawatts, which cost on the order of US$2 mil­lion per month in elec­tric­ity charges. The com­bined pro­cess­ing power of these servers might reach from 20 to 100 petaflops.^[9]

Spec­i­fi­ca­tions:

• Upwards of 15,000 servers^[2] rang­ing from 533 MHz Intel Celeron to dual 1.4 GHz Intel Pen­tium III (as of 2003). A 2005 esti­mate by Paul Strass­mann has 200,000 servers,^[10] while unspec­i­fied sources claimed this num­ber to be upwards of 450,000 in 2006.^[11]
• One or more 80 GB hard disks per server (2003)
• 2–4 GB of mem­ory per machine (2004)

The exact size and where­abouts of the data cen­ters Google uses are unknown, and offi­cial fig­ures remain inten­tion­ally vague. In a 2000 esti­mate, Google’s server farm con­sisted of 6,000 proces­sors, 12,000 com­mon IDE disks (2 per machine, and one proces­sor per machine), at four sites: two in Sil­i­con Val­ley, Cal­i­for­nia and one in Vir­ginia.^[12] Each site had an OC-48 (2488 Mbit/s) inter­net con­nec­tion and an OC-12 (622 Mbit/s) con­nec­tion to other Google sites. The con­nec­tions are even­tu­ally routed down to 4 × 1 Gbit/s lines con­nect­ing up to 64 racks, each rack hold­ing 80 machines and two Eth­er­net switches. The servers run cus­tom server soft­ware called Google Web Server.

Hard­ware details con­sid­ered sensitive

In a 2008 book,^[13] reporter Ran­dall Stross wrote: “Google’s exec­u­tives have gone to extra­or­di­nary lengths to keep the company’s hard­ware hid­den from view. The facil­i­ties are not open to tours, not even to mem­bers of the press.” He wrote this based on inter­views with staff mem­bers and his expe­ri­ence of vis­it­ing the company.

Data cen­ters

Google has numer­ous data cen­ters scat­tered around the world. At least 12 sig­nif­i­cant Google data cen­ter instal­la­tions are located in the United States. The largest known cen­ters are located in The Dalles, Oregon; Atlanta, Georgia; Reston, Virginia; Lenoir, North Car­olina; and Goose Creek, South Car­olina.^[14] In Europe, the largest known cen­ters are inEemshaven and Gronin­gen in the Nether­lands and Mons, Belgium.^[14] Google’s Ocea­nia Data Cen­ter is claimed to be located in Sydney, Australia. ^[15]

Project 02

One of the largest Google data cen­ters is located in the town of The Dalles, Ore­gon, on the Colum­bia River, approx­i­mately 80 miles from Port­land. Code­named “Project 02″, the new com­plex is approx­i­mately the size of two foot­ball­fields, with cool­ing tow­ers four sto­ries high.^[16] The site was cho­sen to take advan­tage of inex­pen­sive hydro­elec­tric power, and to tap into the region’s large sur­plus of fiber optic cable, a rem­nant of the dot-com boom. A blue­print of the site has appeared in print.^[17]

Summa paper­mill

In Feb­ru­ary 2009, Stora Enso announced that they had sold the Summa paper mill in Hamina, Finland to Google for 40 mil­lion Euros.^[18][19] Google plans to invest 200 mil­lion euros on the site to build a data cen­ter.^[20]

Soft­ware

Most of the soft­ware stack that Google uses on their servers was devel­oped in-house.^[21] It is believed that C++, Java, and Python are favored over other pro­gram­ming lan­guages.^[22] Google has acknowl­edged that Python has played an impor­tant role from the begin­ning, and that it con­tin­ues to do so as the sys­tem grows and evolves.^[23]

The soft­ware that runs the Google infra­struc­ture includes:^[24]

• Google Web Server
• Google File System
• BigTable
• Chubby lock service
• MapRe­duce and Sawzall pro­gram­ming language
• Pro­to­col buffers

Server oper­a­tion

Most oper­a­tions are read-only. When an update is required, queries are redi­rected to other servers, so as to sim­plify con­sis­tency issues. Queries are divided into sub-queries, where those sub-queries may be sent to dif­fer­ent ducts inpar­al­lel, thus reduc­ing the latency time.^[2]

To lessen the effects of unavoid­able hard­ware fail­ure, soft­ware is designed to be fault tol­er­ant. Thus, when a sys­tem goes down, data is still avail­able on other servers, which increases reliability.

Catchup

Apple has been build­ing a MDC (Mas­sive Data Cen­ter) in Maiden North Car­olina, and is said to have gone into oper­a­tion already, but Apple has not made clear on how it is using it at this time. The 500,000 square foot facil­ity is five times larger than the Newark Cal­i­for­nia facil­ity (109,500 square foot, bought from World­Com @ 45 mil­lion, a bar­gain con­sid­er­ing it cost 110 mil­lion to build) it owns, is said to cost Apple more than 1 Bil­lion US dol­lars to build.

Named the iDat­a­cen­ter, many spec­u­late this site to house the cloud based iTunes and other ser­vices that Apple plans to deliver in the future.  With it’s own Newark and Cuper­tino data cen­ter sup­ported by addi­tional ser­vices from Aka­mai and Lime­light, why does it have to spend 1 bil­lion USD in North Car­olina? (The 1 bil­lion USD price tag is about twice what Micosoft and Google spend for their data cen­ter.) The answer is fairly sim­ple, it get a larger tax incen­tive from the state if a com­pany invests more than 1 bil­lion dol­lars over 9 years.

It is said that Eric Schmidt stole all the secrets from Apple dur­ing his days as a board mem­ber of Apple, but know­ing Steve Jobs, I  have a feel­ing that Steve would have picked Eric’s brains on how to con­struct, run and use a data cen­ter as well. I find it unlikely that Apple would launch such an aggres­sive invest­ment (yes, Apple’s data cen­ter is the biggest and most expen­sive in the cor­po­rate world) into data cen­ter if it did not have con­fi­dence on con­struc­tion, oper­a­tion, usage, return and investment.

For What?

On June 7th, 2010, Apple may unveil a new ser­vice of sorts that use the data cen­ter. Apple pur­chase of Lala and Quat­tro may have a direct rela­tion to the MDCs.  I am sure Apple needs iDat­a­cen­ter just to ful­fill the App  and iTunes store sales, so many more may be in the pipeline.

Many spe­cial­ists feel that data cen­ter is so large and close to Pen­ta­gon grade, that usage of this power can not be filled by iTunes, SaaS, Lala, Quat­tro, iPhone, iBook­store or the App store.

In an arti­cle in Cult of the Mac, they inter­view Rich Miller who is the edi­tor of Data Cen­ter Knowl­edge, an online trade mag­a­zine devoted to the data cen­ter industry.

CoM: First, any idea why Apple is build­ing this new data center?

Miller: Apple has said very lit­tle about the North Car­olina facil­ity, beyond the fact that it will serve as the company’s East coast data hub. Apple also has a West coast data cen­ter facil­ity in Newark, Calif. Local offi­cials I’ve spo­ken with say they believe the space is pri­mar­ily to sup­port Mobile Me and dig­i­tal con­tent for the iTunes store. The most inter­est­ing ques­tion is whether Apple needs a much larger facil­ity to sup­port growth in its exist­ing ser­vices, or is scal­ing up capac­ity for future offerings.

CoM: Could Apple be build­ing it for cloud com­put­ing apps — cloud ver­sions of its iLife apps for example?

Miller: One of the lead­ing the­o­ries about the size of the NC project is that Apple is plan­ning future cloud com­put­ing ser­vices that will require lots of data cen­ter stor­age. Cloud com­put­ing is a hot trend, and I’d be sur­prised if Apple isn’t think­ing hard – and think­ing dif­fer­ently – about cloud com­put­ing. Many cloud enthu­si­asts say that cloud com­put­ing will elim­i­nate the need for data cen­ters. In real­ity, the only thing will change is the owner of the build­ing. All the appli­ca­tions and data that are mov­ing into the cloud will live on servers in brick-and-mortar data cen­ters. The com­pa­nies that are build­ing the biggest data cen­ters tend to also have the biggest cloud ambitions.

CoM: How big is Apple’s new North Car­olina data cen­ter — big, small, medium?

Miller: The early site plans indi­cate Apple is plan­ning about 500,000 square feet of data cen­ter space in a sin­gle build­ing. That would place it among the largest data cen­ters in the world. For com­par­i­son pur­poses, Apple’s exist­ing data cen­ter in Newark, Calif. is a lit­tle more than 100,000 square feet. Most new stand-alone enter­prise data cen­ters are in the range of 100,000 to 200,000 square feet. So this would qual­ify as a big-ass data center.

CoM: What’s it com­pa­ra­ble to? Do you know of any spe­cific examples?

Miller: In the past sev­eral years we’ve seen a hand­ful of new facil­i­ties that are redefin­ing the scope of mod­ern data cen­ters. These include Microsoft’s new facil­ity in Chicago, the Super­NAP in Las Vegas and the Phoenix ONE colo­ca­tion cen­ter in Phoenix. All of these facil­i­ties house at least 400,000 square feet of space. These data cen­ters are designed to sup­port an enor­mous vol­ume of data, and reflect the accel­er­a­tion of the tran­si­tion to a dig­i­tal econ­omy. All those dig­i­tal assets – email, images, video and now vir­tual machines – drive demand for more and larger data centers.

CoM: Why did Apple chose NC? Are there par­tic­u­larly big pipes in NC? A big pow­er­plant nearby?

Miller: The choice of rural North Car­olina sug­gests that the bot­tom line for Apple is cost, rather than con­nec­tiv­ity. The site in Maiden, NC is not far from a large data cen­ter by Google, which usu­ally chases cheap power and tax incen­tives. Power from Duke Energy is about 4 to 5 cents per kilo­watt hour, com­pared to 7 to 12 cents in Cal­i­for­nia. The com­pany also max­i­mized its incen­tives by pit­ting Vir­ginia and North Car­olina against one another in try­ing to wring the best tax incen­tives out of both states (a pop­u­lar strat­egy in data cen­ter site location).

Some large com­pa­nies use dis­trib­uted data cen­ters to man­age their latency and con­tent deliv­ery costs. That may be part of Apple’s think­ing, since they’re a major cus­tomer for CDNs (I believe they use both Aka­mai and Lime­light Net­works). Face­book cited latency to Europe as a key fac­tor in its deci­sion to add data cen­ters in Vir­ginia. Before that, MySpace added a data cen­ter in Los Ange­les to reduce its reliance on CDNs. But in both cases, those com­pa­nies sought out Inter­net hubs where they could con­nect with dozens of other net­works to man­age their Inter­net traf­fic. You don’t get that in rural North Car­olina, soApple seems more focused on cost and scale than on con­nec­tiv­ity – which again would sug­gest a cloud focus.

Unfo­tu­nartluy this inter­view does not reveal or hint what Apple may use the data cen­ter for, only to spec­u­late that it will han­dle a lot of data.

Lot’s of Data? (IMPORTANT READ HERE)

Yes, lots. What uses a lot of data? Hum, video comes to mind. With the iPhone 4G unmasked and it’s frontal cam­era and 5mega pixel rear cam­era (Apple is also said to have 8mega pixel cam­era ver­sion in the field), and con­sid­er­ing that it is an inno­va­tor in dig­i­tal lifestyle appli­ca­tion, it is not far fetched to think that Apple may have a new “Video Social Net­work” planned that may put Face­book, Twit­ter, Foursquare, Ustream and You Tube to rest. With Vimeo qual­ity HD video (as iPhone most likely will be named “iPhone HD” for a rea­son, not fad), face and loca­tion recog­ni­tion, AR and AR (Arti­fi­cial Real­ity and Aug­mented Real­ity), this would be a killer offer­ing if it came true. Apple with it’s new iPhone and SaaS may store every­ones video, tag them, add face, voice and text recog­ni­tion to them and use this data to cre­ate and sug­gest rela­tion­ship with peo­ple. Apple may have more infor­ma­tion on it’s users that FBI’s col­lected data over the decades.

Video is the miss­ing link in SOCIAL. Yes­ter­day I did an event and streamed it live. I had 4,410 view­ers (2,638 unique) of this show. I am just amazed that some­one like me goes live and you get 4k peo­ple watch­ing the show at one time or the other. Twit­ter of course enhances Ustream, the plat­form which was used for the event. All I needed was a Mac­Book Pro note­book and 2 video cam­eras, 4 micro­phones and a audio mixer.

VIDEO is KING of ALL THINGS DIGITAL

I said this and I will say this again, there is noth­ing to clear and easy to under­stand than mov­ing pic­ture. If 2 peo­ple read a book, their expe­ri­ences may be com­pletely dif­fer­ent. With mov­ing pic­tures it is always the same. If geo tag­ging, short mes­sages, moods, AR and AR can be part of ” what are you going now” twit­ter post­ing, then there will be no more need for any of the SNS ser­vices which are now frag­mented and dif­fi­cult to under­stand. You Tube and Ustream is show­ing Apple how good a poten­tial busi­ness this is, what You Tube and Ustream lack in ease of use and depth of con­tent since nei­ther makes hard­ware and oper­at­ing systems.

Apple has been in a unique posi­tion to watch closely the devel­op­ments of VIDEO SOCIAL (although they are only avail­able in frag­ments and encom­pass mul­ti­ple ser­vices and hard­ware and oper­at­ing sys­tem plat­form thus caus­ing chaos to some users), it is sin­gle­hand­edly in a posi­tion to offer a seam­less solu­tion via it’s OS, hard­ware and deliv­ery plat­form. One but­ton and your done with video, geo and text.

If Apple really wants to embark on video, some ana­lysts say, they may need more of these 1 bil­lion dol­lar data cen­ters around the world to fight latency and sheer num­ber of uploads each iPhone HD will store.

Who else ?

Ora­cle and the US gov­ern­ment are also speed­ing up the con­struc­tion of MDCs in 2010.  Ora­cle stopped con­struc­tion of the MDC in Salt Lake City but then again has recently resumed work on the 240,000 square foot 285 mil­lion dol­lar project. Oracle’s main busi­ness is CRM SaaS (soft­ware as ser­vice) and needs these cen­ters to take on more speed and cus­tomer demand. MDCs are being built in Lon­don, Wales, Tokyo, Tsukuba and many other cities around the world.

Make no mis­take about it, Apple and Google are  at war, but come June 7th, Apple may throw in some new weapons to com­bat Googles announce­ments that it made at Google I/O, or it may not, since it may announce a ser­vice so ambi­tious and close to our daily life, our method of shar­ing notched up to a higher level. In such case, oth­ers will play catch up with new ser­vice offer­ings and data cen­ters, but once Apple takes the lead, it may be hard for Google to orga­nize HTC and other man­u­fac­tur­ers to have a com­mon user inter­face prod­uct to pro­vide the seam­less user expiri­ence pro­vided by a com­pany that makes everything.

Ever since 1985, I have been involved with Apple, not because I am an Apple freak (in hon­esty I am but), but because it makes good busi­ness sense to trust a com­pany that has a solid vision (not dur­ing Job’s absence I must say), owns and can con­trol the direc­tion it wants to go. To me the HTC Desire is like a Fer­rari 430 with a Toy­ota engine,  while the Nexus One is like a Fer­rari Cal­i­for­nia with a Nis­san engine, not bad but in the end, i will no longer buy Fer­rari since what I am buy­ing is nei­ther a Fer­rari, Toy­ota or a Nissan.

It may be a really good time to be in a con­struc­tion com­pany or a com­pany mak­ing con­tainer unit mod­ule for data centers.