Ever wondered how the weather affects your Parkrun running performance?

When's a good day to try for a parkrun PB? Obviously when you're feeling 100%. How about when it's not windy? Yes, obviously. But then, what about atmospheric pressure: surely that would make a difference? With high pressure, you get more oxygen into your lungs for the same amount of breath and we all know that elite athletes train at altitude in order to get peak performance on return to sea level. So, do the smaller day-to-day changes in barometric pressure make a measurable difference?

After some unexpected fluctuations in some of my recent parkrun and training run performances, I thought it was worth trying to find out whether the barometer could confirm what I have always suspected. The availability of parkrun data and historical weather records online makes it relatively easy. So, I gathered the numbers**. Specifically: the proportion of New PBs in parkrun events. I started picking the data from my local parkrun at Bedford* from 2020 to mid-January 2023. I also collected the barometric pressure and wind speed for 9am on those Saturdays. That gave me 87 data points and the Pearson (straight line) correlation was more convincing than I had expected (r=0.31, p=0.003). That means it's just a 3 in 1000 chance that there's not really any correlation. If you're trying to interpret the "r" value, you just need to know that the variables were pressure in hPa against percent of finishers with a new PB. 

As we all get tiresomely reminded: "correlation is not causation". Well, in this case, it's really hard to see it any other way! There's no way that running performance can affect the weather. It's equally inconceivable that a third factor affects both running performance and the weather. The only remaining possibility is that the weather affects something else that in turn affects running performance. OK. That's plausible. It could be that only the more dedicated runners turn out on days when the weather is looking bad and those dedicated runners have mostly run dozens or even hundreds of parkruns. They are less likely to get a new PB than those doing their third run. One day, I'll re-run the data gathering and collect the extra data that I would need to disprove this. I think that I already know that it's not a significant factor because the turnout numbers for my local parkrun are pretty much unaffected by the weather.

Next, I collected the same data for 16 other locations. I picked the larger parkruns because bigger datasets mean less noise. Now with about 1500 data points we get r=0.263 p=2.20e-24 which means it's as good as conclusive. 

The highest dot on the chart award goes to BlythLinks parkrun on 4 Sept 2021 when over 30% of runners got a PB. Awesome!

But hold on!

Wind is another important factor and the complication is that it tends to be associated with low pressure. Maybe all of the effect that I am seeing is due to wind. So, I tried largely reducing the wind factor by excluding data points where the wind speed was more than 25 KPH. That does affect the results: r=0.212, p=1.59e-13 but it still leaves a ridiculously strong correlation. I went a bit further than that by getting a coefficient for the effect of wind and then using that to adjust each data point to remove the effect of wind. Again, it reduces the effect to a small extent leaving, still, a crazy-strong correlation (r=0.163 p=4.66e-10).

Is a straight-line correlation the right thing to try? Theoretically no, because you can never get less than zero percent PBs or more than 100 so it cannot be a linear relationship. However, within the range of pressure seen over this period, the straight line looks like a good enough approximation.

Next time you're wondering whether today is the day to really push it, have a look at the weather forecast. If the pressure is over 1020 then it could well by your PB day. Overall, your chance of getting a PB increases by about 1 percentage point per 10 mmHg increase in pressure.

* Bedford parkrun course is particularly well-sheltered from the wind. The higher correlation here than elsewhere is possibly because the wind factor is naturally excluded.

**

# Python 3.6 minimum
# Program to correlate PBs in a parkrun against weather conditions wind speed and pressure
# My theory is that runners perform better in high pressure.  Download weather data first and rename to {town}.csv 
#   where {town} is the name of the parkrun e.g. "poole". 
# 
# 1. For weather data, start at https://open-meteo.com/en/docs/historical-weather-api#latitude=52.13&
#   longitude=-0.47&start_date=2022-01-01&end_date=2023-01-20&hourly=surface_pressure,windspeed_10m
# 2. Put the place name in the "select city" box, then click on "download csv"
# 3. Rename your csv from "archive.csv" to "{town}.csv"
# 4. Run this script like: python runner_pb.py {town}
# 5. When you have collected from plenty of parkruns, you see the combined data using: 
     python runner_pb.py all
# 6. Output for the graph is "rungraph.csv"

import os, requests, datetime, time, sys, glob
import numpy as np
import scipy.stats

def placename(x):  # map function
    return x.replace("_runners.txt", "")
    
def main():

    try:
        location = sys.argv[1]
    except:
        print("Run this with a location name or 'all' on the command line, e.g. python runner_pb.py bedford")
        sys.exit()

    if location == "all":
        locations = glob.glob("*_runners.txt")
        locations = map( placename, locations) 
    else:
        locations = [location]
        collected = os.path.isfile(location+"_runners.txt")

        if not collected:
            last, weather = collect_weather_data(location) # a dict like: date = {[9am_pressure, 9am_wind]}

            run = collect_run_data(location, last)   # a dict like date = {[pbs_count, runners_count]}
            
            with open(location + "_runners.txt", "w", encoding="utf-8") as fi:
                for rundate, rundata in run.items():
                    wd = weather.get(rundate)
                    if wd:
                        percentage = rundata[0]/rundata[1] * 100.0
                        fi.write(f'{rundate.strftime("%Y-%m-%d")} {percentage:5.2f}% {wd[0]} {wd[1]}\n')

    pb_array = []
    pressure_array = []
    wind_array = []
    wx_pressure_array = []
    wx_pb_array = []
    wp_pr_array = []
    nowind_pr_array = []
    
    for location in locations:

        with open(location+"_runners.txt","r", encoding="utf-8") as fi:
            lines = fi.read().split("\n")

        for l in lines:
            if not l: continue
            l = l.replace("  "," ")
            values  = l.split(" ")
            pb = values[1].replace("%","")
            pressure = values[2]
            wind = values[3]
            pb_array.append(float(pb))
            pressure_array.append(float(pressure))
            wind_array.append(float(wind))
        
    y = np.array(pressure_array)
    x = np.array(pb_array)
    r,p = scipy.stats.pearsonr(x, y)
    print(f"Pressure Correlation: {r:.3f}, {p:.2e}")
    y = np.array(wind_array)
    r,p = scipy.stats.pearsonr(x, y)
    print(f"Wind Correlation: {r:.3f}, {p:.2e}")

    for i in range(len(pressure_array)):
        if wind_array[i] < 25:
            wx_pressure_array.append(pressure_array[i])
            wx_pb_array.append(pb_array[i])
        wp_pr_array.append(pressure_array[i] - 0.69 * wind_array[i])  # -0.69 was found by trial and error as the 
        # wind factor additive to pressure.
        # to get the steepest slope and lowest p-value. Doing +0.69*wind should totally negate the effect of wind 
        nowind_pr_array.append(pressure_array[i] + 0.69 * wind_array[i])
    y = np.array(wx_pressure_array)
    x = np.array(wx_pb_array)
    r,p = scipy.stats.pearsonr(x, y)
    print(f"Wind-suppressed Pressure Correlation: {r:.3f}, {p:.2e}")
    y = np.array(nowind_pr_array)
    x = np.array(pb_array)
    r,p = scipy.stats.pearsonr(x, y)
    print(f"Wind-excluded Pressure Correlation: {r:.3f}, {p:.2e}")
    y = np.array(wp_pr_array)
    x = np.array(pb_array)
    r,p = scipy.stats.pearsonr(x, y)
    print(f"Wind-added Pressure Correlation: {r:.3f}, {p:.2e}")

    with open("rungraph.csv","w") as fi:
        fi.write('"pressure","windspeed","pressure-wind","pb percentage"\n')
        for i in range(len(pressure_array)):
            fi.write(f'{pressure_array[i]},{wind_array[i]},{wp_pr_array[i]},{pb_array[i]}\n')
    with open("xygraph.csv","w") as fi:
        fi.write('"pressure","pb percentage"\n')
        for i in range(len(pressure_array)):
            fi.write(f'{pressure_array[i]},{pb_array[i]}\n')
    
def collect_weather_data(location):  # this comes from a file downloaded from https://open-meteo.com/en/docs
	# /historical-weather-api#latitude=52.13&longitude=-0.47&start_date=2022-01-01&end_date=
    # 2023-01-20&hourly=surface_pressure,windspeed_10m
    # the file has to be renamed as {location}.csv in the same folder as here
    
    retval = {}
    first = None
    
    with open(location + '.csv','r', encoding='utf-8') as fi:
        lines = fi.read().split("\n")
    for line in lines[4:]:
        if len(line) < 5 or line[4] != '-': continue
        if not "09:00" in line: continue
        
        weatherdate = datetime.datetime.strptime( line[:10],'%Y-%m-%d')
        if not first: first = weatherdate
        fields = line.split(",")
        pressure = float(fields[1])
        wind = float(fields[2])
        
        retval[weatherdate] = (pressure,wind)
        
    return first, retval

def collect_run_data(location,last):  
    retval = {}
    starter = f'https://www.parkrun.org.uk/{location}/results/'
    url = starter + "latestresults/"
    headers={'User-Agent': '''Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 
    (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36'''}

    while True:
        response = requests.get(url, headers=headers)
        if response.status_code != 200:
            print(f"Page fetch error {response.status_code} collecting run data")
            return
        page = response.text
    
        # assimilate the data
        rundate = datetime.datetime.strptime(findbetween(page,'<span class="format-date">','<'),'%d/%m/%Y')
        if rundate < last: 
            break
        pb = page.count('data-achievement="New PB!"')
        runners = page.count('Results-table-td--ageGroup')
        
        retval[rundate] = (pb, runners)
        # set the next url
        pageno = int( findbetween(page,'</span><span>#','<')) - 1
        print(pageno)
        time.sleep(1)
        url = starter + str(pageno) 
        
    return retval
    
def findbetween(page,tag1,tag2,after=None,multi=False):

    results = []
    if not page:
        if multi: return []
        return ""
        
    start = 0
    if after:
        start = page.find(after)
        if start < 0: 
            if multi:
                return []
            else:
                return ""
    while 1:
        starting = page.find(tag1,start)
        if starting < 0: 
           if multi: return results
           else: return ""
        l1 = len(tag1)
        ending = page.find(tag2,starting + l1)
        if ending == -1: result = page[starting+l1:]
        else: result = page[starting+l1:ending]
        if not multi: return result
        start = ending
        results.append(result)
    return results
    


if __name__ == "__main__": main()