Consumer Energy Report is now Energy Trends Insider -- Read More »

By Robert Rapier on Sep 10, 2011 with 23 responses

Un-Analytics: How Google Went Solar

While I don’t normally put up a post on Saturday, the queue is piling up a bit so I decided to squeeze this one in before Monday’s post.

The following guest essay is by Dan Auld. Dan is a San Diego businessman and writer whose work has appeared in hundreds of places and who has won several awards from San Diego press associations.


Un-Analytics: How Google Went Solar

September 8, 2011

Google loves talking about the world before analytics — when web owners knew almost nothing about their sites.

Nothing useful, anyway.

That all changed when a new technology came along that allowed web owners to monitor their sites as much as they wanted. Any time they wanted.

Web sites suddenly became a business proposition, not just an enthusiasm for a few hobbyists.

Flash forward from the introduction of Google Analytics up to 2007, when Google got into the solar business and opened a 1.65 megawatt photovoltaic power array. The largest commercial system in the world at the time.

Just like web sites before analytics, Google would soon learn how little it actually knew about its solar array.

After its panels were up for 15 months, Google cleaned them and documented its efforts in a report called “Getting the most energy out of Google’s solar panels.”

On several sections of its array, solar energy output doubled after the cleaning. Eight months later, energy output went up 37 percent after another cleaning. But here comes the money paragraph:

It would be difficult to detect manufacturer defects or accidental damage by data analysis alone, unless the damage impacts >~20% of the solar panels in that building.

Example: There have been few occasions when some of the solar panels … were damaged by delivery trucks accidentally hitting the support beams that hold up the solar panels.

Since these accidents did not damage a sizable portion of the solar panels, the damage went undetected for a while.

Losing 50 percent of your power is real money, even for Google.

“Just like the web prior to analytics, Google had to admit it really did not know what was happening in its array — because it had no way to monitor when good panels went bad,” said Mark Yarbrourgh, a city councilman in Perris, California who pioneered the use of solar in public buildings. “But neither does anyone else. Arrays malfunction and no one knows because they do not use monitors at the panel level.”

Undetected, solar panels go bad in all sorts of ways. Panels degrade anywhere from .5% to 9.5% a year, depending on the manufacturer, says Sandia Laboratories in a study for the Department of Energy.

How will you know what your panels will do? Warranty Week Magazine says you won’t. Not really:

“And yes, it really is guesswork.”

Dirt plays even more havoc. If not dirt, a bird dropping, or a baseball, or a golfball, or a rock, or a squirrel chewing a wire, or a Texas oak thick with pollen, or heat on the roof, or poor soldering. Or a shadow — all worse than you think, says the National Renewable Energy Laboratories:

“ The reduction in power from shading half of one cell is equivalent to removing a cell active area 36 times the shadow’s actual size.”

“One bird, one truck of dirt, one flowering tree can destroy your solar production, and you would not know for a long time,” Yarbrough said. “Welcome to the Christmas Tree Effect: Hurt the panel a little, hurt production a lot. It is amazing how many people put up solar for great reasons, but really do not watch their systems. As a result, a lot of people lose a lot of money because many, many systems are not producing the power its owners were promised. And few know.”

Maybe because knowing it is not that useful.

“If your solar array produces a megawatt of power, that means it is composed of 3000 to 5000 panels,” said Ray Burgess, CEO of Solar Power Technologies. “If some panels go bad, you need panel level monitoring to find the bad panels. But most systems monitor power at the system level, but as Google found out, that is that useful for detecting catastrophic failure, but not much else.”

Thus the need for small wireless monitors throughout the array.

“Now that we have cost effective monitors from a company in Austin, that is going to change the world, just like Google Analytics.”

Leading the solar monitor business is Burgess and Solar Power Technologies of Austin, Texas. The company is introducing monitors and other devices to give solar array owners unprecedented control over their panels. If you have 3500 panels and a few start breaking, you better have something better than “guesswork” to optimize your array.

“As we travel the country talking to panel owners about their systems, we are constantly amazed at how many systems that are producing power far below their capacity, and some not producing power at all. Monitors on the panels can change that and let you know what is really happening with your system. And where it is happening. Saving system owners thousands of dollars a month.”

Just like Google Analytics.

  1. By rufus on September 10, 2011 at 11:28 pm

    Damn, this is just common sense. This industry really is “in its infancy.”

    This is partly what happens with a sudden influx of huge government subsidies. A “Gold Rush” mentality takes hold. A lot of “details,” and “small things” get overlooked.

    The attitude is sorta, “we’ll get the money now, and come back and deal with the loose ends, later.” And, actually, the system does work. But, sometimes you look back, and laugh in amazement, at what went on “back in the Golden days.” :)

  2. By rufus on September 10, 2011 at 11:31 pm

    There’s no reason in the world why every panel shouldn’t be reporting it “status” continuously. And, of course, soon they will.

  3. By Wendell Mercantile on September 11, 2011 at 12:07 am

    Undetected, solar panels go bad in all sorts of ways. Panels degrade anywhere from .5% to 9.5% a year, depending on the manufacturer, says Sandia Laboratories…

    Degrades as much as 9.5% per year? Wow, that’s a surprising stat.

  4. By moiety on September 11, 2011 at 3:51 am

    Rufus said:

    There’s no reason in the world why every panel shouldn’t be reporting it “status” continuously. And, of course, soon they will.


    There is, cost. In the process industries installing equipment like say a distillation tower can be expensive. However setting up and installing all the alarms and controllers on that system is always a significant cost (1-5% level) and in some cases can be much higher. Having a dedicated device for each panel is overkill (why not have a device for each cell?); maybe a device for a number of panels is the way to go.

    Wendell Mercantile said:

    Degrades as much as 9.5% per year? Wow, that’s a surprising stat.


    I have heard that before for thin film but there are relatively few freely available articles that show long term data for several cells. The idea behind that is that cell technology changes so quickly that any data would be obsolete. However that is now changing with commercial technologies. In my opinion the differences in manufacturers is not degradation rate but efficiency (low quality material versus high quality). I tend to think of degradation as being between technologies i.e. thin film will have much lower lifetimes than crystalline.

  5. By Frank Weigert on September 11, 2011 at 7:58 am

    “Dan Auld. Dan is a San Diego businessman and writer”

    Who EXACTLY does he work for?

  6. By rufus on September 11, 2011 at 8:43 am

    “Lifetime,” indeed, looks like the “Big” question with thin-film.

    There is still a lot to “shake out” in this industry.

    That’s why, as I told Wendell, I couldn’t possibly recommend to a friend, or local board, buying Solar, “Today.” I want to see at least one more year (maybe a couple.)

  7. By robert on September 11, 2011 at 10:40 am

    Panels degrade anywhere from .5% to 9.5% a year, depending on the manufacturer, says Sandia Laboratories…

    No, Sandia doesn’t say that. They said the panels degrade 9.5% in the first year. And I can’t find the results they site still on the web. I can’t find the date this paper was published. Nothing in the bibliography is after 2001.

    Here’s a study by the same company.…..e-tiso.pdf

    Initial degradation occurs during the first three months only. Average -1.1% and ranges from -0.6% to -2.9%.

    The declared power of a panel is a number the manufacturer warrantees.
    Warrantee week magazine “and yes it’s really guesswork” refers to manufacturer trying to figure out how much to reserve for warrantee claims. When you look at the data, the resrves are about 1% of sales and claims came in around 1% of sales. They didn’t guess too badly.

  8. By Kit P on September 11, 2011 at 11:30 am

    Rufus get confused sometimes. The goal is to make electricity not sell junk. It takes a different skill set to produce power than to sell stuff that does not work. The reason FP&L is a leader in making electricity with solar is because they apply the same skill set for making electricity with nukes.

    “There is, cost.”

    Exactly correct! Replacement power for a large nuke cost is $1-2 million a day depending on the time of year. I can justify a lot of sensors on critical pumps.

    Replacement power for solar panels on Google’s roof cost is $2-4 thousand a day

    Replacement power for a home roof PV system cost is $5-10 a day.

    The scam artist will tell that PV does not break however, when it does break replacement parts are expensive and technicians to fix electricity generating equipment cost $30-40/hr.

    I do not have any statistics to support the concept but the only PV systems get repaired are large utility scale PV systems operated by people that already have a staff of technicians.

  9. By John Wallace on September 11, 2011 at 5:13 pm

    Well, very interesting. Both the article, which tells the truth about how little we know about the operating results of existing solar power systems; and a few of the reactions: THIS MUST BE SOME KIND OF PLOT!

    So here is the question: If you own a megawatt, or even a bigger rooftop. And you see you production is down, how would you know why it is happening?

    You don’t.

    And say you knew it was a panel, and you had anywhere from 200 to 10,000 panels. How would you know which one was bad.

    You would not.

    So get some monitors, already.

  10. By rufus on September 11, 2011 at 5:30 pm

    Yeah, po ol’ Rufus does gets confused, sometimes; but he can read.

    And, when he clicked on the link he read where Sandia, in an 8 yr, outdoors test, definitely came up with the number 0.005/yr.

    Hint: look for the big, bold type – Module Degradation

  11. By rufus on September 11, 2011 at 5:33 pm

    Some car company touts that its car makes 2.5 Million decisions/second, or somesuch. I’m sure it will be very cheap to monitor the wattage per individual panel.

  12. By robert on September 11, 2011 at 10:05 pm

    Solar power technologies touts their product for commercial and utility scale installations. I have 18 panels on my house. If you’re talking about a10,000 panel installation producing millions of dollars of electricity per year, as Kitp points out you can somebody working full time optimizing the output of your system. Even improving performance by 1% is real money.

    I’m sure it will be very cheap to monitor the wattage per individual panel.

    The problem is that panels are installed in strings and measuring the current in each panel doesn’t tell you which panel is bad. Say you have a string of christmas lights connected in series and the whole string is dark, that doesn’t tell you which bulb is bad.

    Then you tie several strings in parallel so the output voltage is the minimum voltage produced by any string. Measuring this voltage doesn’t tell you which string is bad. I think trying to figure out which panel is bad by remote sensing is trickier than it sounds.

  13. By rufus on September 11, 2011 at 10:25 pm

    In my limited experience, most everything is “trickier than it sounds.”

    somebody’ll figure it out, though. There’s too much money involved.

  14. By Wendell Mercantile on September 11, 2011 at 10:31 pm

    …most everything is “trickier than it sounds.”…somebody’ll figure it out, though. There’s too much money involved.


    That’s what they’ve been saying about fusion power for the last 50 years.

  15. By moiety on September 12, 2011 at 3:25 am

    robert said:

    Panels degrade anywhere from .5% to 9.5% a year, depending on the manufacturer, says Sandia Laboratories…

    No, Sandia doesn’t say that. They said the panels degrade 9.5% in the first year. And I can’t find the results they site still on the web. I can’t find the date this paper was published. Nothing in the bibliography is after 2001.

    Here’s a study by the same company.…..e-tiso.pdf

    Initial degradation occurs during the first three months only. Average -1.1% and ranges from -0.6% to -2.9%.



    Those are for crystalline and the numbers are not unexpected (though a few months is not a long time).

  16. By Tim Johnson on September 12, 2011 at 10:41 am

    Rufus said…

    There’s no reason in the world why every panel shouldn’t be reporting it “status” continuously. And, of course, soon they will.

    tenKsolar’s RAIS PV modules have on board diagnostics, greatly aiding field service. And they degrate 5 x slower, and the impact due to shading is 36 times better than conventional modules. The solution really is different.

  17. By Brian Ballek on September 12, 2011 at 12:32 pm

    Guys, monitoring PV systems is really not that hard. Every module has a unique identifier — a so called “Flash ID”. When a large system is built there’s always one or more little shacks where the inverters convert from DC to AC. If the system has been built correctly, in that room sits a laptop with a bit of software that knows how many strings are running, where they are, and which Flash IDs belong to each string. The laptop communicates by wireless modem with a control center, (operated either by the installer or by a third party monitoring service) and if production on a sunny day drops for longer than it takes a cloud to fly by, alarm bells ring and a remote technician makes a maintenance call, but quick.

    With residential systems, it’s harder to justify the cost of monitoring, but just like all other PV system components, that cost is falling each year. Most inverter manufacturers are already beginning to build web servers into their devices. Once configured, these will regularly upload your systems’ output to a monitoring service, that can check your system’s output against the amount of sun your 5-9 digit zipcode got on any given day. Any drop in performance not accompanied by a drop in sun levels is noted and if it continues, the system owner gets a notification. It’s still (in my opinion) to expensive for systems under 30 kWp but it will be built into inverters for all systems before too long.

    Finding the actual module that is damaged isn’t so hard either. Damaged cells (and modules with damaged cells) may look perfectly OK to the naked eye but NOT to an infrared camera. Those buggers are indeed very pricey (as are the technicians who carry them) but they can find the defective module in one or more strings without too much effort.

    Once the module is found there is a bit of a harry problem: if the replacement module is let’s say 10 years newer than the defective module, it is almost certainly more efficient, producing more power than other modules in the same string. Since the system was designed for a particular output level per string, this is problematic. This is one of the reasons several companies (among them National Semiconductor) are now offering mini-inverters to be integrated into the back of *each* module. If this technology is perfected, one will be able to pull and replace modules in a PV system (large or small) even mixing modules from different suppliers and I think even different module technologies (crystalline, amorphous, CIS, cadmium telluride or whatever) within the same string. There are still issues, particularly regarding module guarantees (for example if 2 modules each with 150W nameplate capacity go south and the current model produces 300W , am I entitled to 2 replacement modules or just one representing the lost capacity?) but these will be worked out with time and competition.

  18. By rrapier on September 12, 2011 at 2:12 pm

    Brian Ballek said:

    Guys, monitoring PV systems is really not that hard.


    I was thinking about this over the weekend, and it certainly seems to me that it wouldn’t be hard to monitor the performance of individual panels. It would seem to me that it would mostly be a software issue.


  19. By paul-n on September 13, 2011 at 1:05 am

    I was thinking about this over the weekend, and it certainly seems to me that it wouldn’t be hard to monitor the performance of individual panels.

    It is indeed very easy, if you have the right equipment, and it is available today!

    For home and small commercial systems, that would be a micro-inverter like the Enphase system.  It puts an inverter, and MPPT(maximum power point tracking) controller on the back of each panel.  The output is 208V AC, 60 Hz, and you can have up to 20 panels connected in series.

    A really good write of a user’s experience with this system is here, including examples of the data charting they do.

    Most Enphase systems are bought with an Envoy or EMU unit that communicates with and monitors the status and performance of each of the micro inverters in the system.  The EMU also sends this performance data up to the Enphase server.  Enphase builds a web page for each system owner.  The web page reports the performance of the system.  It also emails you if there are problems with the system.  The page provides quite detailed performance data (right down to the individual PV panel) and saves all the data so that historical performance can be reviewed.

    One advantage of this systm is that is does MPPT(maximum power point tracking) of each panel, not just the whole array, so the power output is optimised further.  And, it can also be used with different panels of different sizes and outputs, from different manufacturers!

    Here is a picture of a partial shading of two panels in a ten panel array (from the build it solar site) ;


    Now that is not a lot of shading, but look at the output of those two panels;

    if the whole array was set up with one inverter, then the output of the entire array would be decreased by up to 40%!

    Just like a real power plant, where you need an operator who knows what they are doing to monitor and optimise the operation, so too with solar.  The service provided by Enphase seems like a pretty good business model for small systems that can;t justify a dedicated operator.

    Enphase is based in Petaluma, California, and as far as I  can tell, have never been the recipient of any gov funding etc.  A rare “clean tech” success story.

    Until the Chinese start mass producing these things, of course…

    I expect that sometime soon, there will be panels made with this sort of system built in, delivering AC output and the data communication.

    Currently the cost is about $1/W for the inverter, and about $600 for the internet connection unit, so a bit more than a normal inverter, but in anything less than ideal conditions, I expect the improved output more than makes up for that.




  20. By russ on September 13, 2011 at 2:20 pm

    There are numerous micro inverters on the market today – Enphase is the biggest and best known.

    They recently introduced a new series and cahnged the warranty period to 25 years to be more or less equal to the panel.

    They also cut the monitoring charges drastically though I don’t remember the numbers.

    If there is any shade issues then a micro inverter is good. The PC display is also impressive so people can play ^show and tell’ with their little friends and neighbors.

    The micro inverters are more expensive today and there is a lot of sales blather about how they improve annual output – blather.

    For commercial installations, NO one wants 1,000 or 10,000 individual inverters to monitor. As someone else pointed out it is not hard to locate problem panels and central inverters are better from a maintence stand point. 

    One consideration for home use is replacement of a micro inverter in case of a problem – roof top arrays are not designed for easy access normally – changing out an individual micro inverter could be painful – a central inverter on the other hand is easy.

    One place where micro inverters win is expected life – a central inverter is expected to be replaced after maybe 10 years.


  21. By paul-n on September 13, 2011 at 4:29 pm



    Agreed that no one wants to monitor 1000+ inverters/panels.  For a system of that size (would be about 200kW) you would like to think it would not have shading issues.

    I did say that the micro inverters are ideal for home and small commercial systems – which I take to mean <10-20kW.   They are the closest thing to plug and play yet available, though I think the AC output panel is not far away.

    As for the annual output, we’ll have to see some annual studies on that – which to date I haven’t, though I have not looked for them either.

    They also make it feasible to grid connect really small systems – i.e. one, two or three panels, which sometimes may be all that the non-shaded south facing roof can fit.  Or, they make it feasible to gradually add on to a small system over time, even with different makes/sizes/locations of panels (e.g a roof of a house and garage that have different orientation and sun conditions).


    This doesn;t mean that small solar is worthwhile doing everywhere – in many/most places it is not.  But for places that have sunny hot summers, and high electricity prices,  like the southern states and Australia, it allows small homeowners to get in on the game.

  22. By Gary Reysa on September 14, 2011 at 10:03 am

    I don’t really see a problem with microinverters for for large installations. When a microinverter fails, the reporting system tells you which one it is, and replacing one is literally a 5 minute job — very simple. They don’t fail very often, so its not like someone would have to be running around replacing them all day. Since the failure of one microinverter only effects one PV module, you could wait until a few have failed and do them all at once — maybe after (say) 0.5% of the inverters had failed and do them all at once — perhaps as a part of other scheduled routine maintenance.
    I think that is more a matter of whether they can be cost competitive on large installations.

  23. By paul-n on September 15, 2011 at 3:44 am

    Hi Gary, and welcome to R-squared/Consumer Energy Report – I am an ethusiastic supporter of your pragmatic approach to all things solar.

    I agree the microinverters are analagous to changing light bulbs, just not that big a deal to do, or to leave one out for a while.  I guess it would be possible to wire up the roof top panels with a “home run” style system, so all the micro inverters can be mounted on a wall, in cabinet, etc.  This would solve the servicing issues, though the simplicity of installation is lost

    I doubt they will be competitive on large projects, as having a dedicated site operator can achive much of what they do.  But their monitoring ability may push the line farther up the scale as to when it is worth having that dedicated operator.

    And if they are able to improve and maximise the performance of the many underperforming small scale installations then that is a step in the right direction.




Register or log in now to save your comments and get priority moderation!