Reconstruction of geomagnetic activity and near-Earth interplanetary conditions over the past 167 yr - Part 4: Near-Earth solar wind speed, IMF, and open solar flux

In the concluding paper of this tetralogy, we here use the different geomagnetic activity indices to reconstruct the near-Earth interplanetary magnetic field (IMF) and solar wind flow speed, as well as the open solar flux (OSF) from 1845 to the present day. The differences in how the various indices vary with near-Earth interplanetary parameters, which are here exploited to separate the effects of the IMF and solar wind speed, are shown to be statistically significant at the 93% level or above. Reconstructions are made using four combinations of different indices, compiled using different data and different algorithms, and the results are almost identical for all parameters. The correction to the aa index required is discussed by comparison with the Ap index from a more extensive network of mid-latitude stations. Data from the Helsinki magnetometer station is used to extend the aa index back to 1845 and the results confirmed by comparison with the nearby St Petersburg observatory. The optimum variations, using all available long-term geomagnetic indices, of the near-Earth IMF and solar wind speed, and of the open solar flux, are presented; all with ±2sigma� uncertainties computed using the Monte Carlo technique outlined in the earlier papers. The open solar flux variation derived is shown to be very similar indeed to that obtained using the method of Lockwood et al. (1999).

Evidence for solar wind modulation of lightning

The response of lightning rates over Europe to arrival of high speed solar wind streams at Earth is investigated using a superposed epoch analysis. Fast solar wind stream arrival is determined from modulation of the solar wind V y component, measured by the Advanced Composition Explorer spacecraft. Lightning rate changes around these event times are determined from the very low frequency arrival time difference (ATD) system of the UK Met Office. Arrival of high speed streams at Earth is found to be preceded by a decrease in total solar irradiance and an increase in sunspot number and Mg II emissions. These are consistent with the high speed stream's source being co-located with an active region appearing on the Eastern solar limb and rotating at the 27 d period of the Sun. Arrival of the high speed stream at Earth also coincides with a small (~1%) but rapid decrease in galactic cosmic ray flux, a moderate (~6%) increase in lower energy solar energetic protons (SEPs), and a substantial, statistically significant increase in lightning rates. These changes persist for around 40 d in all three quantities. The lightning rate increase is corroborated by an increase in the total number of thunder days observed by UK Met stations, again persisting for around 40 d after the arrival of a high speed solar wind stream. This result appears to contradict earlier studies that found an anti-correlation between sunspot number and thunder days over solar cycle timescales. The increase in lightning rates and thunder days that we observe coincides with an increased flux of SEPs which, while not being detected at ground level, nevertheless penetrate the atmosphere to tropospheric altitudes. This effect could be further amplified by an increase in mean lightning stroke intensity that brings more strokes above the detection threshold of the ATD system. In order to remove any potential seasonal bias the analysis was repeated for daily solar wind triggers occurring during the summer months (June to August). Though this reduced the number of solar wind triggers to 32, the response in both lightning and thunder day data remained statistically significant. This modulation of lightning by regular and predictable solar wind events may be beneficial to medium range forecasting of hazardous weather.

The interdependence of continental warm cloud properties derived from unexploited solar background signals in ground-based lidar measurements

We have extensively analysed the interdependence between cloud optical depth, droplet effective radius, liquid water path (LWP) and geometric thickness for stratiform warm clouds using ground-based observations. In particular, this analysis uses cloud optical depths retrieved from untapped solar background signals that are previously unwanted and need to be removed in most lidar applications. Combining these new optical depth retrievals with radar and microwave observations at the Atmospheric Radiation Measurement (ARM) Climate Research Facility in Oklahoma during 2005–2007, we have found that LWP and geometric thickness increase and follow a power-law relationship with cloud optical depth regardless of the presence of drizzle; LWP and geometric thickness in drizzling clouds can be generally 20–40 % and at least 10 % higher than those in non-drizzling clouds, respectively. In contrast, droplet effective radius shows a negative correlation with optical depth in drizzling clouds and a positive correlation in non-drizzling clouds, where, for large optical depths, it asymptotes to 10 μm. This asymptotic behaviour in non-drizzling clouds is found in both the droplet effective radius and optical depth, making it possible to use simple thresholds of optical depth, droplet size, or a combination of these two variables for drizzle delineation. This paper demonstrates a new way to enhance ground-based cloud observations and drizzle delineations using existing lidar networks.

Influence of short-term solar disturbances on the fair weather conduction current

The fair weather atmospheric electrical current (Jz) couples the ionosphere to the lower atmosphere and thus provides a route by which changes in solar activity can modify processes in the lower troposphere. This paper examines the temporal variations and spectral characteristics of continuous measurements of Jz conducted at the Wise Observatory in Mitzpe-Ramon, Israel (30°35′ N, 34°45′ E), during two large CMEs, and during periods of increased solar wind density. Evidence is presented for the effects of geomagnetic storms and sub-storms on low latitude Jz during two coronal mass ejections (CMEs), on 24–25th October 2011 and 7–8th March 2012, when the variability in Jz increased by an order of magnitude compared to normal fair weather conditions. The dynamic spectrum of the increased Jz fluctuations exhibit peaks in the Pc5 frequency range. Similar low frequency characteristics occur during periods of enhanced solar wind proton density. During the October 2011 event, the periods of increased fluctuations in Jz lasted for 7 h and coincided with fluctuations of the inter-planetary magnetic field (IMF) detected by the ACE satellite. We suggest downward mapping of ionospheric electric fields as a possible mechanism for the increased fluctuations.

Using reanalysis data to quantify extreme wind power generation statistics : a 33 year case study in Great Britain

With a rapidly increasing fraction of electricity generation being sourced from wind, extreme wind power generation events such as prolonged periods of low (or high) generation and ramps in generation, are a growing concern for the efficient and secure operation of national power systems. As extreme events occur infrequently, long and reliable meteorological records are required to accurately estimate their characteristics. Recent publications have begun to investigate the use of global meteorological “reanalysis” data sets for power system applications, many of which focus on long-term average statistics such as monthly-mean generation. Here we demonstrate that reanalysis data can also be used to estimate the frequency of relatively short-lived extreme events (including ramping on sub-daily time scales). Verification against 328 surface observation stations across the United Kingdom suggests that near-surface wind variability over spatiotemporal scales greater than around 300 km and 6 h can be faithfully reproduced using reanalysis, with no need for costly dynamical downscaling. A case study is presented in which a state-of-the-art, 33 year reanalysis data set (MERRA, from NASA-GMAO), is used to construct an hourly time series of nationally-aggregated wind power generation in Great Britain (GB), assuming a fixed, modern distribution of wind farms. The resultant generation estimates are highly correlated with recorded data from National Grid in the recent period, both for instantaneous hourly values and for variability over time intervals greater than around 6 h. This 33 year time series is then used to quantify the frequency with which different extreme GB-wide wind power generation events occur, as well as their seasonal and inter-annual variability. Several novel insights into the nature of extreme wind power generation events are described, including (i) that the number of prolonged low or high generation events is well approximated by a Poission-like random process, and (ii) whilst in general there is large seasonal variability, the magnitude of the most extreme ramps is similar in both summer and winter. An up-to-date version of the GB case study data as well as the underlying model are freely available for download from our website: http://www.met.reading.ac.uk/~energymet/data/Cannon2014/.

Open solar flux estimates from near-Earth measurements of the interplanetary magnetic field: comparison of the first two perihelion passes of the Ulysses spacecraft

Results from all phases of the orbits of the Ulysses spacecraft have shown that the magnitude of the radial component of the heliospheric field is approximately independent of heliographic latitude. This result allows the use of near- Earth observations to compute the total open flux of the Sun. For example, using satellite observations of the interplanetary magnetic field, the average open solar flux was shown to have risen by 29% between 1963 and 1987 and using the aa geomagnetic index it was found to have doubled during the 20th century. It is therefore important to assess fully the accuracy of the result and to check that it applies to all phases of the solar cycle. The first perihelion pass of the Ulysses spacecraft was close to sunspot minimum, and recent data from the second perihelion pass show that the result also holds at solar maximum. The high level of correlation between the open flux derived from the various methods strongly supports the Ulysses discovery that the radial field component is independent of latitude. We show here that the errors introduced into open solar flux estimates by assuming that the heliospheric field’s radial component is independent of latitude are similar for the two passes and are of order 25% for daily values, falling to 5% for averaging timescales of 27 days or greater. We compare here the results of four methods for estimating the open solar flux with results from the first and second perehelion passes by Ulysses. We find that the errors are lowest (1–5% for averages over the entire perehelion passes lasting near 320 days), for near-Earth methods, based on either interplanetary magnetic field observations or the aa geomagnetic activity index. The corresponding errors for the Solanki et al. (2000) model are of the order of 9–15% and for the PFSS method, based on solar magnetograms, are of the order of 13–47%. The model of Solanki et al. is based on the continuity equation of open flux, and uses the sunspot number to quantify the rate of open flux emergence. It predicts that the average open solar flux has been decreasing since 1987, as Correspondence to: M. Lockwood (m.lockwood@rl.ac.uk) is observed in the variation of all the estimates of the open flux. This decline combines with the solar cycle variation to produce an open flux during the second (sunspot maximum) perihelion pass of Ulysses which is only slightly larger than that during the first (sunspot minimum) perihelion pass.

An evaluation of the correlation between open solar flux and total solar irradiance

The correlation between the coronal source flux F_{S} and the total solar irradiance I_{TS} is re-evaluated in the light of an additional 5 years' data from the rising phase of solar cycle 23 and also by using cosmic ray fluxes detected at Earth. Tests on monthly averages show that the correlation with F_{S} deduced from the interplanetary magnetic field (correlation coefficient, r = 0.62) is highly significant (99.999%), but that there is insufficient data for the higher correlation with annual means (r = 0.80) to be considered significant. Anti-correlations between I_{TS} and cosmic ray fluxes are found in monthly data for all stations and geomagnetic rigidity cut-offs (r ranging from −0.63 to −0.74) and these have significance levels between 85% and 98%. In all cases, the t is poorest for the earliest data (i.e., prior to 1982). Excluding these data improves the anticorrelation with cosmic rays to r = −0:93 for one-year running means. Both the interplanetary magnetic field data and the cosmic ray fluxes indicate that the total solar irradiance lags behind the open solar flux with a delay that is estimated to have an optimum value of 2.8 months (and is within the uncertainty range 0.8-8.0 months at the 90% level).

Ground-based and satellite observations of high-latitude auroral activity in the dusk sector of the auroral oval

On 7 December 2000, during 13:30-15:30 UT the MIRACLE all-sky camera at Ny Alesund observed auroras at high-latitudes (MLAT similar to 76) simultaneously when the Cluster spacecraft were skimming the magnetopause in the same MLT sector (at similar to 16:00-18:00 MLT). The location of the auroras (near the ionospheric convection reversal boundary) and the clear correlation between their dynamics and IMF variations suggests their close relationship with R1 currents. Consequently, we can assume that the Cluster spacecraft were making observations in the magnetospheric region associated with the auroras, although exact magnetic conjugacy between the ground-based and satellite observations did not exist. The solar wind variations appeared to control both the behaviour of the auroras and the magnetopause dynamics. Auroral structures were observed at Ny Alesund especially during periods of negative IMF B-Z. In addition, the Cluster spacecraft experienced periodic (T similar to 4 - 6 min) encounters between magnetospheric and magnetosheath plasmas. These undulations of the boundary can be interpreted as a consequence of tailward propagating magnetopause surface waves. Simultaneous dusk sector ground-based observations show weak, but discernible magnetic pulsations (Pc 5) and occasionally periodic variations (T - 2 - 3 min) in the high-latitude auroras. In the dusk sector, Pc 5 activity was stronger and had characteristics that were consistent with a field line resonance type of activity. When IMF BZ stayed positive for a longer period, the auroras were dimmer and the spacecraft stayed at the outer edge of the magnetopause where they observed electromagnetic pulsations with T similar to 1 min. We find these observations interesting especially from the viewpoint of previously presented studies relating poleward-moving high-latitude auroras with pulsation activity and MHD waves propagating at the magnetospheric boundary layers.

Coordinated ground-based, low altitude satellite and Cluster observations on global and local scales during a transient post-noon sector excursion of the magnetospheric cusp

On 14 January 2001, the four Cluster spacecraft passed through the northern magnetospheric mantle in close conjunction to the EISCAT Svalbard Radar (ESR) and approached the post-noon dayside magnetopause over Greenland between 13:00 and 14:00 UT During that interval, a sudden reorganisation of the high-latitude dayside convection pattern accurred after 13:20 UT most likely caused by a direction change of the Solar wind magnetic field. The result was an eastward and poleward directed flow-channel, as monitored by the SuperDARN radar network and also by arrays of ground-based magnetometers in Canada, Greenland and Scandinavia. After an initial eastward and later poleward expansion of the flow-channel between 13:20 and 13:40 UT, the four Cluster spacecraft, and the field line footprints covered by the eastward looking scan cycle of the Sondre Stromfjord incoherent scatter radar were engulfed by cusp-like precipitation with transient magnetic and electric field signatures. In addition, the EISCAT Svalbard Radar detected strong transient effects of the convection reorganisation, a poleward moving precipitation, and a fast ion flow-channel in association with the auroral structures that suddenly formed to the west and north of the radar. From a detailed analysis of the coordinated Cluster and ground-based data, it was found that this extraordinary transient convection pattern, indeed, had moved the cusp precipitation from its former pre-noon position into the late post-noon sector, allowing for the first and quite unexpected encounter of the cusp by the Cluster spacecraft. Our findings illustrate the large amplitude of cusp dynamics even in response to moderate solar wind forcing. The global ground-based data proves to be an invaluable tool to monitor the dynamics and width of the affected magnetospheric regions.

Coordinated Cluster, ground-based instrumentation and low-altitude satellite observations of transient poleward-moving events in the ionosphere and in the tail lobe

During the interval between 8:00-9:30 on 14 January 2001, the four Cluster spacecraft were moving from the central magnetospheric lobe, through the dusk sector mantle, on their way towards intersecting the magnetopause near 15:00 MLT and 15:00 UT. Throughout this interval, the EIS-CAT Svalbard Radar (ESR) at Longyearbyen observed a series of poleward-moving transient events of enhanced F-region plasma concentration ("polar cap patches"), with a repetition period of the order of 10 min. Allowing for the estimated solar wind propagation delay of 75 ( 5) min, the interplanetary magnetic field (IMF) had a southward component during most of the interval. The magnetic footprint of the Cluster spacecraft, mapped to the ionosphere using the Tsyganenko T96 model (with input conditions prevailing during this event), was to the east of the ESR beams. Around 09:05 UT, the DMSP-F12 satellite flew over the ESR and showed a sawtooth cusp ion dispersion signature that also extended into the electrons on the equatorward edge of the cusp, revealing a pulsed magnetopause reconnection. The consequent enhanced ionospheric flow events were imaged by the SuperDARN HF backscatter radars. The average convection patterns (derived using the AMIE technique on data from the magnetometers, the EISCAT and SuperDARN radars, and the DMSP satellites) show that the associated poleward-moving events also convected over the predicted footprint of the Cluster spacecraft. Cluster observed enhancements in the fluxes of both electrons and ions. These events were found to be essentially identical at all four spacecraft, indicating that they had a much larger spatial scale than the satellite separation of the order of 600 km. Some of the events show a correspondence between the lowest energy magnetosheath electrons detected by the PEACE instrument on Cluster (10-20 eV) and the topside ionospheric enhancements seen by the ESR (at 400-700 km). We suggest that a potential barrier at the magnetopause, which prevents the lowest energy electrons from entering the magnetosphere, is reduced when and where the boundary-normal magnetic field is enhanced and that the observed polar cap patches are produced by the consequent enhanced precipitation of the lowest energy electrons, making them and the low energy electron precipitation fossil remnants of the magnetopause reconnection rate pulses.

Blowing in the solar wind

When the solar wind blows: The northern lights are a sign of the awesome power that the Earth receives from the solar wind. The big puzzle is how

The ionospheric signatures of flux transfer events and solar wind dynamic pressure changes

The generation of flow and current vortices in the dayside auroral ionosphere has been predicted for two processes ocurring at the dayside magnetopause. The first of these mechanisms is time-dependent magnetic reconnection, in “flux transfer events” (FTEs); the second is the action of solar wind dynamic pressure changes. The ionospheric flow signature of an FTE should be a twin vortex, with the mean flow velocity in the central region of the pattern equal to the velocity of the pattern as a whole. On the other hand, a pulse of enhanced or reduced dynamic pressure is also expected to produce a twin vortex, but with the central plasma flow being generally different in speed from, and almost orthogonal to, the motion of the whole pattern. In this paper, we make use of this distinction to discuss recent observations of vortical flow patterns in the dayside auroral ionosphere in terms of one or other of the proposed mechanisms. We conclude that some of the observations reported are consistent only with the predicted signature of FTEs. We then evaluate the dimensions of the open flux tubes required to explain some recent simultaneous radar and auroral observations and infer that they are typically 300 km in north–south extent but up to 2000 km in longitudinal extent (i.e., roughly 5 hours of MLT). Hence these observations suggest that recent theories of FTEs which invoke time-varying reconnection at an elongated neutral line may be correct. We also present some simultaneous observations of the interplanetary magnetic field (IMF) and solar wind dynamic pressure (observed using the IMP8 satellite) and the ionospheric flow (observed using the EISCAT radar) which are also only consistent with the FTE model. We estimate that for continuously southward IMF ( ≈ 5 nT) these FTEs contribute about 30 kV to the mean total transpolar voltage (∼30%).

Investigation of the accuracy of historical irradiance products and interannual variability of solar irradiance using Met Office ground data

Met Office station data from 1980 to 2012 has been used to characterise the interannual variability of incident solar irradiance across the UK. The same data are used to evaluate four popular historical irradiance products to determine which are most suitable for use by the UK PV industry for site selection and system design. The study confirmed previous findings that interannual variability is typically 3–6% and weighted average probability of a particular percentage deviation from the mean at an average site in the UK was calculated. This weighted average showed that fewer than 2% of site-years could be expected to fall below 90% of the long-term site mean. The historical irradiance products were compared against Met Office station data from the input years of each product. This investigation has found that all products perform well. No products have a strong spatial trend. Meteonorm 7 is most conservative (MBE = −2.5%), CMSAF is most optimistic (MBE = +3.4%) and an average of all four products performs better than any one individual product (MBE = 0.3%)

Systematic and random errors between collocated satellite ice water path observations

There remains large disagreement between ice-water path (IWP) in observational data sets, largely because the sensors observe different parts of the ice particle size distribution. A detailed comparison of retrieved IWP from satellite observations in the Tropics (!30 " latitude) in 2007 was made using collocated measurements. The radio detection and ranging(radar)/light detection and ranging (lidar) (DARDAR) IWP data set, based on combined radar/lidar measurements, is used as a reference because it provides arguably the best estimate of the total column IWP. For each data set, usable IWP dynamic ranges are inferred from this comparison. IWP retrievals based on solar reflectance measurements, in the moderate resolution imaging spectroradiometer (MODIS), advanced very high resolution radiometer–based Climate Monitoring Satellite Applications Facility (CMSAF), and Pathfinder Atmospheres-Extended (PATMOS-x) datasets, were found to be correlated with DARDAR over a large IWP range (~20–7000 g m -2 ). The random errors of the collocated data sets have a close to lognormal distribution, and the combined random error of MODIS and DARDAR is less than a factor of 2, which also sets the upper limit for MODIS alone. In the same way, the upper limit for the random error of all considered data sets is determined. Data sets based on passive microwave measurements, microwave surface and precipitation products system (MSPPS), microwave integrated retrieval system (MiRS), and collocated microwave only (CMO), are largely correlated with DARDAR for IWP values larger than approximately 700 g m -2 . The combined uncertainty between these data sets and DARDAR in this range is slightly less MODIS-DARDAR, but the systematic bias is nearly an order of magnitude.

On the importance of Vaisala RS92 radiosonde humidity corrections for a better agreement between measured and modeled satellite radiances

A study has been carried out to assess the importance of radiosonde corrections in improving the agreement between satellite and radiosonde measurements of upper-tropospheric humidity. Infrared [High Resolution Infrared Radiation Sounder (HIRS)-12] and microwave [Advanced Microwave Sounding Unit (AMSU)-18] measurements from the NOAA-17 satellite were used for this purpose. The agreement was assessed by comparing the satellite measurements against simulated measurements using collocated radiosonde profiles of the Atmospheric Radiation Measurement (ARM) Program undertaken at tropical and midlatitude sites. The Atmospheric Radiative Transfer Simulator (ARTS) was used to simulate the satellite radiances. The comparisons have been done under clear-sky conditions, separately for daytime and nighttime soundings. Only Vaisala RS92 radiosonde sensors were used and an empirical correction (EC) was applied to the radiosonde measurements. The EC includes correction for mean calibration bias and for solar radiation error, and it removes radiosonde bias relative to three instruments of known accuracy. For the nighttime dataset, the EC significantly reduces the bias from 0.63 to 20.10 K in AMSU-18 and from 1.26 to 0.35 K in HIRS-12. The EC has an even greater impact on the daytime dataset with a bias reduction from 2.38 to 0.28 K in AMSU-18 and from 2.51 to 0.59 K in HIRS-12. The present study promises a more accurate approach in future radiosonde-based studies in the upper troposphere.