# My Data Kraken – a Shapeshifter

I wonder if Data Kraken is only used by German speakers who translate our hackneyed Datenkrake – is it a word like eigenvector?

Anyway, I need this animal metaphor, despite this post is not about facebook or Google. It’s about my personal Data Kraken – which is a true shapeshifter like all octopuses are:

(… because they are spineless, but I don’t want to over-interpret the metaphor…)

Data Kraken’s shapeability is a blessing, given ongoing challenges:

When the Chief Engineer is fighting with other intimidating life-forms in our habitat, he focuses on survival first and foremost … and sometimes he forgets to inform the Chief Science Officer about fundamental changes to our landscape of sensors. Then Data Kraken has to be trained again to learn how to detect if the heat pump is on or off in a specific timeslot. Use the signal sent from control to the heat pump? Or to the brine pump? Or better use brine flow and temperature difference?

It might seem like a dull and tedious exercise to calculate ‘averages’ and other performance indicators that require only very simple arithmetics. But with the exception of room or ambient temperature most of the ‘averages’ just make sense if some condition is met, like: The heating water inlet temperature should only be calculated when the heating circuit pump is on. But the temperature of the cold water, when the same floor loops are used for cooling in summer, should not be included in this average of ‘heating water temperature’. Above all, false sensor readings, like 0, NULL or any value (like 999) a vendor chooses to indicate as an error, have to be excluded. And sometimes I rediscover eternal truths like the ratio of averages not being equal to the average of ratios.

The Chief Engineer is tinkering with new sensors all the time: In parallel to using the old & robust analog sensor for measuring the water level in the tank…

… a multitude of level sensors was evaluated …

… until finally Mr. Bubble won the casting …

… and the surface level is now measured via the pressure increasing linearly with depth. For the Big Data Department this means to add some new fields to the Kraken database, calculate new averages … and to smoothly transition from the volume of ice calculated from ruler readings to the new values.

Change is the only constant in the universe, paraphrasing Heraclitus [*]. Sensors morph in purpose: The heating circuit, formerly known (to the control unit) as the radiator circuit became a new wall heating circuit, and the radiator circuit was virtually reborn as a new circuit.

I am also guilty of adding new tentacles all the time, too, herding a zoo of meters added in 2015, each of them adding a new log file, containing data taken at different points of time in different intervals. This year I let Kraken put tentacles into the heat pump:

But the most challenging data source to integrate is the most unassuming source of logging data: The small list of the data that The Chief Engineer had recorded manually until recently (until the advent of Miss Pi CAN Sniffer and Mr Bubble). Reason: He had refused to take data at exactly 00:00:00 every single day, so learned things I never wanted to know about SQL programming languages to deal with the odd time intervals.

To be fair, the Chief Engineer has been dedicated at data recording! He never shunned true challenges, like a legendary white-out in our garden, at the time when measuring ground temperatures was not automated yet:

Long-term readers of this blog know that ‘elkement’ stands for a combination of nerd and luddite, so I try to merge a dinosaur scripting approach with real-world global AI Data Krakens’ wildest dream: I wrote scripts that create scripts that create scripts [[[…]]] that were based on a small proto-Kraken – a nice-to-use documentation database containing the history of sensors and calculations.

The mutated Kraken is able to eat all kinds of log files, including clients’ ones, and above all, it can be cloned easily.

I’ve added all the images and anecdotes to justify why an unpretentious user interface like the following is my true Christmas present to myself – ‘easily clickable’ calculated performance data for days, months, years, and heating seasons.

… and diagrams that can be changed automatically, by selecting interesting parameters and time frames:

The major overhaul of Data Kraken turned out to be prescient as a seemingly innocuous firmware upgrade just changed not only log file naming conventions and publication scheduled but also shuffled all the fields in log files. My Data Kraken has to be capable to rebuild the SQL database from scratch, based on a documentation of those ever changing fields and the raw log files.

_________________________________

[*] It was hard to find the true original quote for that, as the internet is cluttered with change management coaches using that quote, and Heraclitus speaks to us only through secondary sources. But anyway, what this philosophy website says about Heraclitus applies very well to my Data Kraken:

The exact interpretation of these doctrines is controversial, as is the inference often drawn from this theory that in the world as Heraclitus conceives it contradictory propositions must be true.

In my world, I also need to deal with intriguing ambiguity!

# Hacking My Heat Pump – Part 2: Logging Energy Values

In the last post, I showed how to use Raspberry Pi as CAN bus logger – using a test bus connected to control unit UVR1611. Now I have connected it to my heat pump’s bus.

Credits for software and instructions:

Special thanks to SK Pang Electronics who provided me with CAN boards for Raspberry Pi after having read my previous post!!

CAN extension boards for Raspberry Pi, by SK Pang. Left: PiCAN 2 board (40 GPIO pins), right: smaller, retired PiCAN board with 26 GPIO pins – the latter fits my older Pi. In contrast to the board I used in the first tests, these have also a serial (DB9) interface.

Wiring CAN bus

We use a Stiebel-Eltron WPF 7 basic heat pump installed in 2012. The English website now refers to model WPF 7 basic s.

The CAN bus connections described in the German manual (Section 12.2.3) and the English manual (Wiring diagram, p.25) are similar:

CAN bus connections inside WPF 7 basic heat pump. For reference, see the description of the Physical Layer of the CAN protocol. Usage of the power supply (BUS +) is optional.

H, L and GROUND wires from the Pi’s CAN board are connected to the respective terminals inside the heat pump. I don’t use the optional power supply as the CAN board is powered by Raspberry Pi, and I don’t terminate the bus correctly with 120 Ω. As with the test bus, wires are rather short and thus have low resistance.

Heat pump with cover removed – CAN High (H – red), Low (L – blue), and Ground (yellow) are connected. The CAN cable is a few meters long and connects to the Raspberry Pi CAN board.

In the first tests Raspberry Pi had the privilege to overlook the heat pump room as the top of the buffer tank was the only spot the WLAN signal was strong enough …

Typical, temporary nerd’s test setup.

… or I used a cross-over ethernet cable and a special office desk:

Typical, temporary nerd’s workplace.

Now Raspberry Pi has its final position on the ‘organic controller board’, next to control unit UVR16x2 – and after a major upgrade to both LAN and WLAN all connections are reliable.

Raspberry Pi with PiCAN board from SK Pang and UVR16x2 control unit from Technische Alternative (each connected to a different CAN bus).

Bringing up the interface

According to messpunkt.org the bit rate of Stiebel-Eltron’s bus is 20000 bit/s; so the interface is activated with:

```sudo ip link set can0 type can bitrate 20000
sudo ifconfig can0 up```

Watching the idle bus

First I was simply watching with sniffer Wireshark if the heat pump says anything without being triggered. It does not – only once every few minutes there are two packets. So I need to learn to talk to it.

SK Pang provides an example of requesting data using open source tool cansend: The so-called CAN ID is followed by # and the actual data. This CAN ID refers to an ‘object’ – a set of properties of the device, like the set of inputs or outputs – and it can contain also the node ID of the device on the bus. There are many CAN tutorials on the net, I found this (German) introduction and this English tutorial very useful.

I was able to follow the communications of the two nodes in my test bus as I knew their node numbers and what to expect – the data logger would ask the controller for a set of configured sensor outputs every minute. Most packets sent by either bus member are related to object 480, indicating the transmission of a set of values (Process Data Exchange Objects, PDOs. More details on UVR’s CAN communication, in German)

Sniffing test CAN bus – communication of UVR1611 (node no 1) and logger BL-NET (node number 62 = be). Both devices use an ID related to object ID 480 plus their respective node number, as described here.

So I need to know object ID(s) and properly formed data values to ask the heat pump for energy readings – without breaking something by changing values.

Collecting interesting heat pump parameters for monitoring

I am very grateful for Jürg’s CAN tool can_scan that allow for querying a Stiebel-Eltron heat pump for specific values and also for learning about all possible parameters (listed in so-called Elster tables).

In order to check the list of allowed CAN IDs used by the heat pump I run:

`./can_scan can0 680`

can0 is the (default) name of the interface created earlier and 680 is my (the sender’s) CAN ID, one of the IDs allowed by can_scan.

Start of output:

```elster-kromschroeder can-bus address scanner and test utility
copyright (c) 2014 Jürg Müller, CH-5524

scan on CAN-id: 680
list of valid can id's:

000 (8000 = 325-07)
180 (8000 = 325-07)
301 (8000 = 325-07)
480 (8000 = 325-07)
601 (8000 = 325-07)```

In order to investigate available values and their meaning I run can_scan for each of these IDs:

`./can_scan can0 680 180`

Embedded below is part of the output, containing some of the values (and /* Comments */). This list of parameters is much longer than the list of values available via the display on the heat pump!

I am mainly interested in metered energies and current temperatures of the heat source (brine) and the ‘environment’ – to compare these values to other sensors’ output:

```elster-kromschroeder can-bus address scanner and test utility
copyright (c) 2014 Jürg Müller, CH-5524

0001:  0000  (FEHLERMELDUNG  0)
0003:  019a  (SPEICHERSOLLTEMP  41.0)
0005:  00f0  (RAUMSOLLTEMP_I  24.0)
0006:  00c8  (RAUMSOLLTEMP_II  20.0)
0007:  00c8  (RAUMSOLLTEMP_III  20.0)
0008:  00a0  (RAUMSOLLTEMP_NACHT  16.0)
0009:  3a0e  (UHRZEIT  14:58)
000a:  1208  (DATUM  18.08.)
000c:  00e9  (AUSSENTEMP  23.3) /* Ambient temperature */
000d:  ffe6  (SAMMLERISTTEMP  -2.6)
000e:  fe70  (SPEICHERISTTEMP  -40.0)
0010:  0050  (GERAETEKONFIGURATION  80)
0013:  01e0  (EINSTELL_SPEICHERSOLLTEMP  48.0)
0016:  0140  (RUECKLAUFISTTEMP  32.0) /* Heating water return temperature */
...
01d4:  00e2  (QUELLE_IST  22.6) /* Source (brine) temperature */
...
/* Hot tap water heating energy MWh + kWh */
/* Daily totaly */
092a:  030d  (WAERMEERTRAG_WW_TAG_WH  781)
092b:  0000  (WAERMEERTRAG_WW_TAG_KWH  0)
/* Total energy since system startup */
092c:  0155  (WAERMEERTRAG_WW_SUM_KWH  341)
092d:  001a  (WAERMEERTRAG_WW_SUM_MWH  26)
/* Space heating energy, MWh + kWh */
/* Daily totals */
092e:  02db  (WAERMEERTRAG_HEIZ_TAG_WH  731)
092f:  0006  (WAERMEERTRAG_HEIZ_TAG_KWH  6)
/* Total energy since system startup */
0930:  0073  (WAERMEERTRAG_HEIZ_SUM_KWH  115)
0931:  0027  (WAERMEERTRAG_HEIZ_SUM_MWH  39)```

Querying for one value

The the heating energy to date in MWh corresponds to index 0931:

`./can_scan can0 680 180.0931`

The output of can_scan already contains the sum of the MWh (0931) and kWh (0930) values:

```elster-kromschroeder can-bus address scanner and test utility
copyright (c) 2014 Jürg Müller, CH-5524

value: 0027  (WAERMEERTRAG_HEIZ_SUM_MWH  39.115)```

The network trace shows that the logger (using ID 680) queries for two values related to ID 180 – the kWh and the MWh part:

Network trace of Raspberry Pi CAN logger (ID 680) querying CAN ID 180. Since the returned MWh value is the sum of MWh and kWh value, two queries are needed. Detailed interpretation of packets in the text below.

Interpretation of these four packets – as explained on Jürg’s website here and here in German:

```00 00 06 80 05 00 00 00 31 00 fa 09 31
00 00 01 80 07 00 00 00 d2 00 fa 09 31 00 27
00 00 06 80 05 00 00 00 31 00 fa 09 30
00 00 01 80 07 00 00 00 d2 00 fa 09 30 00 73
|---------| ||          |---| || |---| |---|
1)          2)          3)    4) 5)    6)

1) CAN-ID used by the sender: 180 or 680
2) No of bytes of data - 5 for queries, 8 for replies
3) CAN ID of the communications partner and type of message.
For queries the second digit is 1.
Pattern: n1 0m with n = 180 / 80 = 3 (hex) and m = 180 mod 7 = 0
(hex) Partner ID = 30 * 8 (hex) + 00 = 180
Responses follow a similar pattern using second digit 2:
Partner ID is: d0 * 8 + 00 = 680
4) fa indicates that the Elster index no is greater equal ff.
5) Index (parameter) queried for: 0930 for kWh and 0931 for MWh
6) Value returned 27h=39,73h=115```

I am not sure which node IDs my logger and the heat pump use as the IDs. 180 seems to be an object ID without node ID added while 301 would refer to object ID + node ID 1. But I suppose with two devices on the bus only, and one being only a listener, there is no ambiguity.

Logging script

I found all interesting indices listed under CAN ID 180; so am now looping through this set once every three minutes with can_scan, cut out the number, and add it to a new line in a text log file. The CAN interfaces is (re-)started every time in case something happens, and the file is sent to my local server via FTP.

Every month a new log file is started, and log files – to be imported into my SQL Server  and processed as log files from UVR1611 / UVR16x2, the PV generator’s inverter, or the smart meter.

(Not the most elegant script – consider it a ‘proof of concept’! Another option is to trigger the sending of data with can_scan and collect output via can_logger.)

Interesting to-be-logged parameters are added to a ‘table’ – a file called indices:

```0016 RUECKLAUFISTTEMP
01d4 QUELLE_IST
01d6 WPVORLAUFIST
091b EL_AUFNAHMELEISTUNG_WW_TAG_KWH
091d EL_AUFNAHMELEISTUNG_WW_SUM_MWH
091f EL_AUFNAHMELEISTUNG_HEIZ_TAG_KWH
0921 EL_AUFNAHMELEISTUNG_HEIZ_SUM_MWH
092b WAERMEERTRAG_WW_TAG_KWH
092f WAERMEERTRAG_HEIZ_TAG_KWH
092d WAERMEERTRAG_WW_SUM_MWH
0931 WAERMEERTRAG_HEIZ_SUM_MWH
000c AUSSENTEMP
0923 WAERMEERTRAG_2WE_WW_TAG_KWH
0925 WAERMEERTRAG_2WE_WW_SUM_MWH
0927 WAERMEERTRAG_2WE_HEIZ_TAG_KWH
0929 WAERMEERTRAG_2WE_HEIZ_SUM_MWH```

Script:

```# Define folders
logdir="/CAN_LOGS"
scriptsdir="/CAN_SCRIPTS"
indexfile="\$scriptsdir/indices"

# FTP parameters
ftphost="FTP_SERVER"
ftpuser="FTP_USER"
ftppw="***********"

if ! [ -d \$scriptsdir ]
then
echo Directory \$scriptsdir does not exist!
exit 1
fi

# Create log dir if it does not exist yet
if ! [ -d \$logdir ]
then
mkdir \$logdir
fi

sleep 5

echo ======================================================================

# Start logging
while [ 0 -le 1 ]
do

# Get current date and start new logging line
now=\$(date +'%Y-%m-%d;%H:%M:%S')
line=\$now
year=\$(date +'%Y')
month=\$(date +'%m')
logfile=\$year-\$month-can-log-wpf7.csv
logfilepath=\$logdir/\$logfile

# Create a new file for every month, write header line
# Create a new file for every month
if ! [ -f \$logfilepath ]
then
do
header=\$(echo \$indexline | cut -d" " -f2)
done < \$indexfile ; echo "\$headers" > \$logfilepath
fi

# (Re-)start CAN interface
sudo ip link set can0 type can bitrate 20000
sudo ip link set can0 up

# Loop through interesting Elster indices
do
# Get output of can_scan for this index, search for line with output values
index=\$(echo \$indexline | cut -d" " -f1)
value=\$(\$scriptsdir/./can_scan can0 680 180.\$index | grep "value" | replace ")" "" | grep -o "\<[0-9]*\.\?[0-9]*\$" | replace "." ",")
echo "\$index \$value"

# Append value to line of CSV file
line="\$line;\$value"
done < \$indexfile ; echo \$line >> \$logfilepath

# echo FTP log file to server
ftp -n -v \$ftphost << END_SCRIPT
ascii
user \$ftpuser \$ftppw
binary
cd RPi
ls
lcd \$logdir
put \$logfile
ls
bye
END_SCRIPT

echo "------------------------------------------------------------------"

# Wait - next logging data point
sleep 180

# Runs forever, use Ctrl+C to stop
done
```

In order to autostart the script I added a line to the rc.local file:

`su pi -c '/CAN_SCRIPTS/pkt_can_monitor'`

Using the logged values

In contrast to brine or water temperature heating energies are not available on the heat pump’s CAN bus in real-time: The main MWh counter is only incremented once per day at midnight. Then the daily kWh counter is added to the previous value.

Daily or monthly energy increments are calculated from the logged values in the SQL database and for example used to determine performance factors (heating energy over electrical energy) shown in our documentation of measurement data for the heat pump system.

# Hacking My Heat Pump – Part 1: CAN Bus Testing with UVR1611

In the old times, measuring data manually sometimes meant braving the elements:

White-Out in winter 2012/13! The barely visible wall is the solar/air collector of our heat pump system.

Measuring ground temperature in different depths.

Now, nearly all measurements are automated:

Online schematic of the heat pump system, showing the temperature and flow sensors needed for control, and a few of the sensors needed for monitoring only (radiation, ground temperature). Screenshot from CMI/UVR1611/UVR16x, Details on system’s operation in this post.

In order to calculate the seasonal performance factor of the heat pump system we have still used the ‘official’ energy reading provided by the heat pump’s display.

Can’t this be automated, too?

Our Stiebel-Eltron WPF7 basic is a simple brine/water heat pump without ‘smart’ features. Our control units turns it on and off via a latch contact.

But there are two interesting interfaces:

• An optical interface to connect a service PC.
• Wired connections to an internal CAN bus – a simple fieldbus used for example in vehicles.

We picked option 2 as it does not require an optical device to read off data. Our control unit also uses CAN bus, and we have test equipment for wired CAN connections.

I always want to use what we already have, and I had a Raspberry Pi not yet put into ‘productive’ use. As usual, you find geeks online who did already what you plan: Reading off CAN bus data provided by a Stiebel-Eltron heat pump using a Raspberry Pi.

In this first post, I am covering the test hardware setup. Before connecting to the heat pump I wanted to test with CAN devices I am familiar with.

Credits

I am indebted to the following sources for information and tools:

On Stiebel-Eltron heat pumps’ CAN bus plus Raspberry Pi

On Raspberry Pi and CAN bus in general / for other applications:

CAN converter

RPi has so-called GPIO pins that let you control devices in the real world. Talking to a CAN device requires an extension board to be connected to these pins.

My challenge: I had the older version – ‘Model B’ – with 26 GPIO pins only. The successor model B Plus had 40 pins. While the pin assignment was not changed, newer CAN extension boards (like this from SK Pang) were too large physically for the old Pi (The older, smaller board from SK Pang had been retired). I was glad to find this small board on ebay.

Edit, 2016-08-24: I replaced the board shown below by SK Pang’s retired PiCAN board – see part 2.

My Pi plus extension board:

CAN extension board connected to the Pi’s GPIO pins and to CAN bus (grey, three wires yellow, red, blue). Black (right) – electrical power, Blue (left): Ethernet. See more info on wiring below in the text.

Wiring the test CAN bus

The image shows the CAN board attached to the Pi, with CAN High, Low, and Ground connected. Following standards, CAN bus needs to be terminated on both ends, using a 120Ω resistor. As our wires are quite short and we had never observed issues with not / falsely terminated short CAN busses so far, we did not add proper termination (BTW: Thanks to ebay seller ZAB for providing the proper resistor!)

In the final setup, the other end of the CAN cable has to be connected to heat pump’s internal bus.

For testing purposes, I am building a CAN bus with three member devices:

1. Test control unit UVR1611 by Technische Alternative. This test unit does not control anything. A single temperature sensor is connected to check if logging works as expected.
2. The unit’s data logger BL-NET: The logger and the control unit communicate via CAN bus and logging data can be transferred to a PC via ethernet. For more details on using control units and loggers by Technische Alternative see this post.
3. My Raspberry Pi plus CAN board – connected to BL-NET.

Middle: Control unit UVR1611 (box with display), one Pt1000 temperature sensor connected (metal tube, black cable), Top: Data logger BL-NET (white box), connected to UVR1611 and Raspberry PI via CAN bus (grey CAN cables, blue plug). The yellow LAN / ethernet cable is for connecting a test PC.

I am using software WinSol on a PC connected via Ethernet to the data logger – to configure logging (BL-NET’s IP address) and to check if the temperature sensor works. BL-NET is set to log data every minute, so that I am sure that CAN packets are available on the bus often. More on WinSol and BL-NET here.

Activating CAN capabilities

Operating system update: I had first used the Raspberry Pi in 2014 using the Raspbian operating system, and I used a pre-installed SD card. Newer versions of the Raspbian Linux operating system do support CAN interfaces, so I just had to upgrade the kernel, described e.g. in CowFish’s instructions (see Software Installation section)

Operating system config: The CAN interface needs the underlying SPI bus – which has to be activated in the Pi’s configuration. This is described in detail on the blog of board vendor SK Pang.

Setting bit rate and bringing up the CAN interface

In order to check if software has been installed correctly, a virtual CAN interface can be configured as a rehearsal:

```sudo modprobe vcan
sudo ip link set vcan0 up```

This interface is not used, so sniffer software (as Wireshark, see below) will not show any communication.

If a physical CAN interface is activated if no CAN bus is physically connected an error cannot find device can0 is expected.

The critical parameter for the physical CAN bus is the bit rate of the bus. For an existing bus, you need to figure out its bit rate from documentation.

According to messpunkt.org the bit rate for the heat pump’s is 20kbit/s. UVR1611’s bus uses bit rate is 50kbit/s, so the interface is configured with

```sudo ip link set can0 type can bitrate 50000
sudo ifconfig can0 up
```

Troubleshooting wrong bit rate

If this is not configured correctly, you will not get errors but you will simply don’t see any packets. Checking the CAN bus (with erroneously configured bit rate) with

`sudo ip -s -d link show can0`

showed that CAN state is BUS OFF …

Inspecting CAN bus performance details, having configured the UVR1611 bus (requiring 50kbit/s) with only 20kbit/s.

… a state the device can enter if there have been too many errors on the bus according to this documentation the CAN protocol family in Linux.

If the bit rate is set to 50000, packets are visible now.

Watching packets flowing by

I’ve installed Wireshark sniffer on the PI…

```sudo apt-get install wireshark
```

… and selected the can0 interface. Packets are flowing, and Wireshark parses them correctly as CAN Protocol!

Network trace of CAN communications on the test CAN bus, consisting of UVR1611 and data logger BL-NET (Talking to each other) plus Raspberry Pi as silent sniffer.

If you know ‘how to speak CAN’ other devices on the bus can be polled for measurement values, using tools, like the Jürg’s CAN Progs or SK Pang’s Test tools linked at the bottom of this article.

In the next post in this series I will cover the setup of the Raspberry Pi CAN sniffer for the heat pump’s CAN bus.

>> Continued >> Part 2

# Half a Year of Solar Power and Smart Metering

Our PV generator and new metering setup is now operational for half a year; this is my next wall of figures. For the first time I am combining data from all our loggers (PV inverter, smart meter for consumption, and heat pump system’s monitoring), and I give a summary on our scrutinizing the building’s electrical power base load.

For comparison: These are data for Eastern Austria (in sunny Burgenland). Our PV generator has 4.77kWp, 10 panels oriented south-east and 8 south-west. Typical yearly energy production in our place, about 48° latitude: ~ 5.300 kWh. In the first 6 months – May to November 2015 – we harvested about 4.000kWh.
Our house (private home and office) meets the statistical average of an Austrian private home, that is about 3.500 kWh/year for appliances (excl. heating, and cooling is negligible here). We heat with a heat pump and need about 7.200kWh electrical energy per year in total.

In the following plots daily and monthly energy balances are presented in three ways:

1. Total consumption of the building as the sum of the PV energy used immediately, and the energy from the utility.
2. The same total consumption as the sum of the heat pump compressor’s input energy and the remaining energy for appliances, computers, control etc.
3. Total energy generated by PV panels as the sum of energy used (same amount as contributing to 1) and the energy sold to the utility.

In summer there is more PV  energy available than needed and – even with a battery – the rest would needed to be fed into the grid. In October, heating season starts and more energy is needed by the heat pump that can be provided by solar energy.

This is maybe demonstrated best by comparing the self-sufficiency quota (ratio of PV energy and energy consumed) and the self-consumption quota (ratio of PV energy consumed and PV production). Number ‘flip’ in October:

In November we had some unusually hot record-breaking days while the weather became more typical at the end of the month:

This is reflected in energy consumption: November 10 was nearly like a summer day, when the heat pump only had to heat hot water, but on the colder day it needed about 20kWh (resulting in 80-100kWh heating energy).

In July, we had the chance to measure what the building without life-forms needs per day – the absolute minimum baseline. On July 10, 11, and 12 we were away and about 4kWh were consumed per day160W on average.

Note that the 4kWh baseline is 2-3 times the energy the heat pump’s compressor needs for hot water heating every day:

We catalogued all devices, googled for data sheets and measured power consumption, flipped a lot of switches, and watched the smart meter tracking the current consumption of each device.

Consumption minus production: Current values when I started to write this post, the sun was about to set. In order to measure the consumption of individual devices they have been switched an of off one after the other, after sunset.

We abandoned some gadgets and re-considered usage. But in this post I want to focus on the base load only – on all devices that contribute to the 160W baseline.

As we know from quantum physics, the observing changes the result of the measurement. It was not a surprise that the devices used for measuring, monitoring and metering plus required IT infrastructure make up the main part of the base load.

Control & IT base load – 79W

• Network infrastructure, telephone, and data loggers – 35W: Internet provider’s DSL modem / router, our router + WLAN access point, switch, ISDN phone network termination, data loggers / ethernet gateways for our control unit, Uninterruptible Power Supply (UPS).
• Control and monitoring unit for the heat pump system, controlling various valves and pumps: 12W.
• The heat pump’s internal control: 10W
• Three different power meters: 22W: 1) Siemens smart meter of the utility, 2) our own smart meter with data logger and WLAN, 3) dumb meter for overall electrical input energy of the heat pump (compressor plus auxiliary energy). The latter needs 8W despite its dumbness.

Other household base load – 39W

• Unobtrusive small gadgets – 12W: Electrical toothbrush, motion detectors, door bell, water softener, that obnoxious clock at the stove which is always wrong and can’t be turned off either, standby energy of microwave oven and of the PV generator’s inverter.
• Refrigerator – 27W: 0,65 kWh per day.

Non-essential IT networking infrastructure – 10W

• WLAN access point and router for the base floor – for connecting the PV inverter and the smart meter and providing WLAN to all rooms.

These are not required 24/7; you don’t lose data by turning them off. Remembering to turn off daily might be a challenge:

Non-24/7 office devices – 21W. Now turned off with a flip switch every evening, and only turned on when needed.

• Scanner/Printer/Fax: 8W. Surprisingly, there was no difference between ‘standby’ and ‘turned off’ using the soft button – it always needs 8W unless you really disconnect it.
• Server in hibernated state 4W. Note that it took a small hack of the operating system already to hibernate the server operating system at all. Years ago the server was on 24/7 and its energy consumption amounted to 500kWh a year.

Stuff retired after this ‘project’ – 16W.

• Radio alarm clock – 5W. Most useless consumption of energy ever. But this post is not meant as bragging about the smartest use of energy ever, but about providing a realistic account backed up by data.
• Test and backup devices – 7W. Backup notebooks, charging all the time, backup router for playground subnet not really required 24/7, timer switch most likely needing more energy than it saved by switching something else.
• Second old Uninterruptable Power Supply – 4W. used for one connected device only, in addition to the main one. It was purchased in the last century when peculiarities of the local power grid had rebooted  computers every day at 4:00 PM.

In total, we were able to reduce the base load by about 40W, 25% of the original value. This does not sound much – it is equivalent to a small light bulb. But on the other hand, it amounts to 350kWh / year, that is 10% of the yearly energy consumption!

___________________________

Logging setup:

• Temperature / compressor’s electrical power: Universal control UVR1611 and C.M.I. as data logger, logging interval 90 seconds. Temperature sensor: PT1000. Power meter:  CAN Energy Meter. Log files are exported daily to CSV files using Winsol. Logging interval: 90 seconds.
• PV output power: Datamanager 2.0 included with PV inverter Fronius Symo 4.5-3-M, logging interval 5 minutes.
• Consumed energy: Smart meter EM-210, logging interval 15 minutes.
• CSV log files are imported into Microsoft SQL Server 2014 for analysis and consolidation. Plots are created with Microsoft Excel as front end to SQL Server, from daily and monthly views on joined UVR1611 / Fronius Symo / EM-210 tables.

# The Impact of Ambient Temperature on the Output Power of Solar Panels

I have noticed the impact of traversing clouds on solar power output: Immediately after a cloud has passed, power surges to a record value. This can be attributed to the focusing effect of the surrounding clouds and/or cooling of the panels. Comparing data for cloudless days in May and June, I noticed a degradation of power – most likely due to higher ambient temperatures in June.

We had a record-breaking summer here; so I wondered if I could prove this effect, using data taken at extremely hot days. There is no sensor on the roof to measure temperature and radiation directly at the panels, but we take data taken every 90 seconds for:

• Ambient air temperature
• Global radiation on a vertical plane, at the position of the solar thermal collector used with the heat pump system.

I was looking for the following:

• Two (nearly) cloudless days, in order to rule out the impact of shadowing at different times of the days.
• These days should not be separated by too many other days, to rule out the effect of the changing daily path of the sun.
• Ideally, air temperature should be very different on these days but global radiation should be the the same.

I found such days: August 1 and August 12 2015:

Daily output of the photovoltaics generator (4,77 kW peak), compared to average and maximum air temperatures and to the global radiation on a vertical plane. Dotted vertical lines indicate three days nearly without clouds.

August 12 was  a record-breaking day with a maximum temperature of 39,5°C. August 1 was one of the ‘cool’ but still perfectly sunny days in August. The ‘cold day’ resulted in a much higher PV output, despite similar input in terms of radiation. For cross-checking I have also included August 30: Still fairly hot, but showing a rather high PV output, at a slightly higher input energy.

August 2015 in detail:

Same data as previous plot, zoomed in on August. Dotted lines indicate the days compared in more detail.

Overlaying the detailed curves for temperature and power output over time for the three interesting days:

Detailed logging of ambient air temperature and output power of the photovoltaic generator on three nearly cloudless days in August 2015.

The three curves are stacked ‘in reverse order’:

The higher the ambient air temperature, the lower the output power.

Note that the effect of temperature can more than compensate for the actually higher radiation for the middle curve (August 30).

I have used global radiation on a vertical plane as an indicator of radiation, not claiming that it is related to the radiation that would be measured on the roof – or on a horizontal plane, as it is usually done – in a simple way. We measure radiation at the position of our ribbed pipe collector that serves as a heat source for the heat pump; it is oriented vertically so that it resembles the orientation of that collector and allows us for using these data as input for our simulations of the performance of the heat pump system.

Our house casts a shadow on the solar collector and this sensor on the afternoon; therefore data show a cut-off in the afternoon:

Global radiation in W per square meter on a vertical plane, measured at the position of the solar collector. The collector is installed on the ground, fence-like, behind the house, about north-east of it.

Yet, if you compare two cloudless days where the sun traversed about the same path (thus days close in the calendar) you can conclude that solar radiation everywhere – including the position on the roof – was the same if these oddly shaped curves are alike.

This plot shows that the curves for these two days that differed a lot in output and temperature, August 1 and 12, were really similar. Actually, the cooler day with higher PV output, August 1, even showed the lower solar radiation due to some spikes. Since the PV inverter only logs every 5 minutes whereas our system’s monitoring logs every 1,5 minutes those spikes might have been averaged out in the PV power curves. August 30 clearly showed higher radiation which can account for the higher output energy. But – as shown above – the higher solar power could not compensate for the higher ambient temperature.

___________________________

Logging setup:

# Heat Pump System Data: Three Seasons 2012 – 2015

We have updated the documentation of monthly and seasonal measurement data – now including also the full season September 2014 to August 2015.

The overall Seasonal Performance Factor was 4,4 – despite the slightly lower numbers in February and March, when was the solar collector was off during the Ice Storage Challenge.

Edit: I have learned from a question that the SPF is also calculated in BTU/Wh. ‘Our’ SPF uses the same units in nominator and denominator, so 4,4 is in Wh/Wh. The conversion factor is about 3,4 (note that I use a decimal comma BTW), so our SPF [kWh/kWh] is equivalent to an SPF [BTU/Wh] ~ 15.

Monthly heating energy provided by the heat pump – total of both space heating and hot water water, related electrical input energy, and the ratio = monthly performance factor. The SPF is in kWh/kWh.

The SPF determines economics of heating with a heat pump.

It’s time to compare costs again, based on current minimum prices of electricity and natural gas in our region in Austria (published by regulator e-control):

• We need about 20.000 kWh (*) of heating energy per year.
• Assuming a nearly perfect gas boiler with an efficiency of 95%, we would need about 21.050 kWh of gas.
• Cost of natural gas incl. taxes, grid fees: ~ 0,0600 € / kWh
• Yearly energy costs for heating with gas would be: € 1.260
• Given an SPF of 4,4 for the heat pump, 20.000 kWh heating energy demands translate to 4.545 kWh of electrical energy.
• Costs of electricity incl. taxes, grid: ~ 0,167 € / kWh
• Yearly energy costs for heating with the heat pump: € 760
• Yearly savings with the heat pump: € 500 or 40% of the costs of gas.

(*) As indicated in the PDF, In the past year only the ground floor was heated by the heat pump. So we needed only 13.300 kWh. In the first floor we got rid of the remainders of the old roof truss. The season 2012/2013 was more typical, requiring about 19.700 kWh.

The last winter was not too extreme – we needed 100 kWh maximum heating energy per day. The collector was capable of harvesting about 50 kWh / day:

Daily energies: 1) Heating energy delivered by the heat pump. Heating energy = electrical energy + ambient energy from the tank. 2) Energy supplied by the collector to the water tank, turned off during the Ice Storage Challenge. Negative collector energies indicate cooling of the water tank by the collector during summer nights. 200 kWh peak in January: due to the warm winter storm ‘Felix’.

Ice formation in this season was mainly triggered by turning off the solar collector deliberately. As soon as we turn the collector on again in March the ice was melted quickly, and the temperature increased to the set value of 8°C – a value picked deliberately to prepare for cooling in summer:

Daily averages of the air temperature and the temperature in the water tank plus volume of ice created by extracting heat from the heat source (water tank).

I am maintaining a list of answers to Frequently Asked Questions here.

# Having Survived the Hottest July Ever (Thanks, Natural Cooling!)

July 2015 was the hottest July ever since meteorological data had been recorded in Austria (since 248 years). We had more than 38°C ambient air temperature at some days; so finally a chance to stress-test our heat pump system’s cooling option.

Heating versus cooling mode

In space heating ‘winter’ mode, the heat pump extracts heat from the heat source – a combination of underground water / ice tank and unglazed solar collector – and heats the bulk volume of the buffer storage tank. We have two heating circuits exchanging heat with this tank – one for the classical old radiators in ground floor, and one for the floor heating loops in the first floor – our repurposed attic.

Space heating mode: The heat pump (1) heats the buffer tank (7), which in turn heats the heating circuits (only one circuit shown, each has its circuit pump and mixer control). Heat source: Solar collector (4) and water / ice storage (3) connected in a single brine circuit. The heat exchanger in the tank is built from the same ribbed pipes as the solar collector. If the ambient temperature is too low too allow for harvesting of energy the 3-way valve (5) makes the brine flow bypass the collector.

The heat pump either heats the buffer tank for space heating, or the hygienic tank for hot tap water. (This posting has a plot with heating power versus time for both modes).

We heat hot tap water indirectly, using a hygienic storage tank with a large internal heat exchanger. Therefore we don’t need to fight legionella by heating to high temperatures, and we only need to heat the bulk volume of the tank to 50°C – which keeps the Coefficient of Performance high.

Hot tap water heating mode: The flow of water heated by the heat pump is diverted to the hygienic storage tank (6). Otherwise, the heat source is used in the same way as for space heating. In this picture, the collector is ‘turned off’ – corresponding to heating water on e.g. a very cold winter evening.

In summer, the still rather cold underground water tank can be used for cooling. Our floor heating loops become cooling loops and we simply use the cool water or ice in the underground tank for natural (‘passive’) cooling. So the heat pump can keep heating water – this is different from systems that turn an air-air heat pump into an air conditioner by reverting the cycle of the refrigerant.

Heating hot water in parallel to cooling is beneficial as the heat pump extracts heat from the underground tank and cools it further!

Cooling mode: Via automated 3-way valve (9) brine is diverted to flow through the heat exchanger in the buffer tank (7). Water in the buffer tank is cooled down so water in the floor ‘heating’ / cooling loops. If the heat pump operates in parallel to heat hot tap water, it cools the brine.

How we optimize cooling power this summer

Water tank temperature. You could tweak the control to keep the large ice cube as long as possible, but there is a the trade-off: The cooler the tank,  the lower the heat pump’s performance factor in heating mode. This year we kept the tank at 8°C after ‘ice season’ as long as possible. To achieve this, the solar collector is bypassed if ambient temperature is ‘too high’. The temperature in the tank rose quickly in April – so our ice is long melted:

The red arrow indicates the end of the ice period; then the set temperature of the tank was 8°C (‘Ice storage tank’ is rather a common term denoting this type of heat source than indicating that it really contains ice all the time.) Green arrows indicate three spells of hot weather. The tank’s temperature increased gradually, being heating by the surrounding ground and by space cooling. At the beginning of August its temperature is close to 20°C, so cooling energy has nearly be used up completely.

At the beginning of July the minimum inlet temperature in the floor loops was 17°C, determined by the dew point (monitored by our control system that controls the mixer accordingly); at the end of the month maximum daily ambient air temperatures were greater than 35°C, and the cooling water had about 21°C.

Room temperature. Cooling was activated only if the room temperature in the 1st floor was higher than 24°C – this allows for keeping as much cooling energy as possible for the really hot periods. We feel that 25°C in the office is absolutely OK as temperatures outside are more then 10°C higher.

Scheduling hot water heating. After the installation of our PV panels we set the hot water heating time slots to periods with high solar radiation – when you have more than 2 kW output power on cloudless days. So we utilized the solar energy generator in the most economic way and the heat pump supports cooling exactly when cooling is needed.

Using the collector for cooling in the night. If the ambient temperature drops to a value lower than the tank temperature, the solar collector can actually cool the tank!

Ventilation. I have been asked if we have forced ventilation, ductwork, and automated awnings etc. No, we haven’t – we just open all the windows during the night and ‘manually operated’ shades attached to the outside of the windows. We call them the Deflector Shields:

Manually operated ventilation – to be shut off at sunrise. We had already 30°C air temperature at 08:00 AM on some days.

South-east deflector shields down. We feel there is still enough light in the (single large) room as we only activate the subset of shields facing the sun directly.

These are details for two typical hot days in July:

The blue line exhibits the cooling power measured for the brine ‘cooling’ circuit. If the heat pump is off, cooling power is about 1 kW; during heat pump operations (blue arrows) 4 kW can be obtained. Night-time ventilation is crucial to keep room temperatures at reasonable levels.

The cooling power is lower than so-called standard cooling load as defined in AC standards – the power required to keep the temperature at about 24°C in steady-state conditions, when ambient temperature would be 30°C and no shades are used. For our attic-office this standard cooling power would amount to more than 10 kW which is higher than the standard (worst case) heating load in winter.

Overall electrical energy balance

I have been asked for a comparison of the energy needed in the house, the heat pump in particular, and the energy delivered by the PV panels and fed in to the grid.

PV numbers in July were not much different from June’s – here is the overview on June and July, maximum PV power on cloudless days has decreased further due to the higher temperatures:

In July, our daily consumption slightly decreased to 9-10 kWh per day, the heat pump needs 1-2 kWh of that. The generator provides for 23 kWh per day,

Currently the weather forecast says, we will have more than 35°C each noon and 20-25° minimum in the night until end of this week. We might experience the utter depletion of our cooling energy storage before it will be replenished again on a rainy next weekend.