Estimating correlated observation error statistics using an ensemble transform Kalman filter

For certain observing types, such as those that are remotely sensed, the observation errors are correlated and these correlations are state- and time-dependent. In this work, we develop a method for diagnosing and incorporating spatially correlated and time-dependent observation error in an ensemble data assimilation system. The method combines an ensemble transform Kalman filter with a method that uses statistical averages of background and analysis innovations to provide an estimate of the observation error covariance matrix. To evaluate the performance of the method, we perform identical twin experiments using the Lorenz ’96 and Kuramoto-Sivashinsky models. Using our approach, a good approximation to the true observation error covariance can be recovered in cases where the initial estimate of the error covariance is incorrect. Spatial observation error covariances where the length scale of the true covariance changes slowly in time can also be captured. We find that using the estimated correlated observation error in the assimilation improves the analysis.

Challenges in quantifying changes in the global water cycle

Understanding observed changes to the global water cycle is key to predicting future climate changes and their impacts. While many datasets document crucial variables such as precipitation, ocean salinity, runoff, and humidity, most are uncertain for determining long-term changes. In situ networks provide long time-series over land but are sparse in many regions, particularly the tropics. Satellite and reanalysis datasets provide global coverage, but their long-term stability is lacking. However, comparisons of changes among related variables can give insights into the robustness of observed changes. For example, ocean salinity, interpreted with an understanding of ocean processes, can help cross-validate precipitation. Observational evidence for human influences on the water cycle is emerging, but uncertainties resulting from internal variability and observational errors are too large to determine whether the observed and simulated changes are consistent. Improvements to the in situ and satellite observing networks that monitor the changing water cycle are required, yet continued data coverage is threatened by funding reductions. Uncertainty both in the role of anthropogenic aerosols, and due to large climate variability presently limits confidence in attribution of observed changes.

Coordinated Cluster/Double Star observations of dayside reconnection signatures

The recent launch of the equatorial spacecraft of the Double Star mission, TC-1, has provided an unprecedented opportunity to monitor the southern hemisphere dayside magnetopause boundary layer in conjunction with northern hemisphere observations by the quartet of Cluster spacecraft. We present first results of one such situation where, on 6 April 2004, both Cluster and the Double Star TC-1 spacecraft were on outbound transits through the dawnside magnetosphere. The observations are consistent with ongoing reconnection on the dayside magnetopause, resulting in a series of flux transfer events (FTEs) seen both at Cluster and TC-1, which appear to lie north and south of the reconnection line, respectively. In fact, the observed polarity and motion of each FTE signature advocates the existence of an active reconnection region consistently located between the positions of Cluster and TC-1, with Cluster observing northward moving FTEs with +/− polarity, whereas TC-1 sees −/+ polarity FTEs. This assertion is further supported by the application of a model designed to track flux tube motion for the prevailing interplanetary conditions. The results from this model show, in addition, that the low-latitude FTE dynamics are sensitive to changes in convected upstream conditions. In particular, changing the interplanetary magnetic field (IMF) clock angle in the model suggests that TC-1 should miss the resulting FTEs more often than Cluster and this is borne out by the observations.

On the cause of a magnetospheric flux transfer event

We present a detailed investigation of a magnetospheric flux transfer event (FTE) seen by the Active Magnetospheric Tracer Explorer (AMPTE) UKS and IRM satellites around 1046 UT on October 28, 1984. This event has been discussed many times previously in the literature and has been cited as support for a variety of theories of FTE formation. We make use of a model developed to reproduce ion precipitations seen in the cusp ionosphere. The analysis confirms that the FTE is well explained as a brief excursion into an open low-latitude boundary layer (LLBL), as predicted by two theories of magnetospheric FTEs: namely, that they are bulges in the open LLBL due to reconnection rate enhancements or that they are indentations of the magnetopause by magnetosheath pressure increases (but in the presence of ongoing steady reconnection). The indentation of the inner edge of the open LLBL that these two models seek to explain is found to be shallow for this event. The ion model reproduces the continuous evolution of the ion distribution function between the sheath-like population at the event center and the surrounding magnetospheric populations; it also provides an explanation of the high-pressure core of the event as comprising field lines that were reconnected considerably earlier than those that are draped over it to give the event boundary layer. The magnetopause transition parameter is used to isolate a field rotation on the boundaries of the core, which is subjected to the tangential stress balance test. The test identifies this to be a convecting structure, which is neither a rotational discontinuity (RD) nor a contact discontinuity, but could possibly be a slow shock. In addition, evidence for ion reflection off a weak RD on the magnetospheric side of this structure is found. The event structure is consistent in many ways with features predicted for the open LLBL by analytic MHD theories and by MHD and hybrid simulations. The de Hoffman-Teller velocity of the structure is significantly different from that of the magnetosheath flow, indicating that it is not an indentation caused by a high-pressure pulse in the sheath but is consistent with the motion of newly opened field lines (different from the sheath flow because of the magnetic tension force) deduced from the best fit to the ion data. However, we cannot here rule out the possibility that the sheath flow pattern has changed in the long interval between the two satellites observing the FTE and subsequently emerging into the magnetosheath; thus this test is not conclusive in this particular case. Analysis of the fitted elapsed time since reconnection shows that the core of the event was reconnected in one pulse and the event boundary layer was reconnected in a subsequent pulse. Between these two pulses is a period of very low (but nonzero) reconnection rate, which lasts about 14 mins. Thus the analysis supports, but does not definitively verify, the concept that the FTE is a partial passage into an open LLBL caused by a traveling bulge in that layer produced by a pulse in reconnection rate.

Causes of plasma flow bursts and dayside auroral transients: An evaluation of two models Invoking reconnection pulses and changes in the Y component of the magnetosheath field

Longitudinal flow bursts observed by the European Incoherent Scatter (EISCAT) radar, in association with dayside auroral transients observed from Svalbard, have been interpreted as resulting from pulses of enhanced reconnection at the dayside magnetopause. However, an alternative model has recently been proposed for a steady rate of magnetopause reconnection, in which the bursts of longitudinal flow are due to increases in the field line curvature force, associated with the By component of the magnetosheath field. We here evaluate these two models, using observations on January 20, 1990, by EISCAT and a 630-nm all-sky camera at Ny Ålesund. For both models, we predict the behavior of both the dayside flows and the 630-nm emissions on newly opened field lines. It is shown that the signatures of steady reconnection and magnetosheath By changes could possibly resemble the observed 630-nm auroral events, but only for certain locations of the observing site, relative to the ionospheric projection of the reconnection X line: however, in such cases, the flow bursts would be seen between the 630-nm transients and not within them. On the other hand, the model of reconnection rate pulses predicts that the flows will be enhanced within each 630-nm transient auroral event. The observations on January 20, 1990, are shown to be consistent with the model of enhanced reconnection rate pulses over a background level and inconsistent with the effects of periodic enhancements of the magnitude of the magnetosheath By component. We estimate that the reconnection rate within the pulses would have to be at least an order of magnitude larger than the background level between the pulses.

Implications of the altitude of transient 630-nm dayside auroral emissions

The altitude from which transient 630-nm (“red line”) light is emitted in transient dayside auroral breakup events is discussed. Theoretically, the emissions should normally originate from approximately 250 to 550 km. Because the luminosity in dayside breakup events moves in a way that is consistent with newly opened field lines, they have been interpreted as the ionospheric signatures of transient reconnection at the dayside magnetopause. For this model the importance of these events for convection can be assessed from the rate of change of their area. The area derived from analysis of images from an all-sky camera and meridian scans from a photometer, however, depends on the square of the assumed emission altitude. From field line mapping, it is shown for both a westward and an eastward moving event, that the main 557.7-nm emission comes from the edge of the 630 nm transient, where a flux transfer event model would place the upward field-aligned current (on the poleward and equatorward edge, respectively). The observing geometry for the two cases presented is such that this is true, irrespective of the 630-nm emission altitude. From comparisons with the European incoherent scatter radar data for the westward (interplanetary magnetic field By > 0) event on January 12, 1988, the 630-nm emission appears to emanate from an altitude of 250 km, and to be accompanied by some 557.7-nm “green-line” emission. However, for a large, eastward moving event observed on January 9, 1989, there is evidence that the emission altitude is considerably greater and, in this case, the only 557.7-nm emission is that on the equatorward edge of the event, consistent with a higher altitude 630-nm excitation source. Assuming an emission altitude of 250 km for this event yields a reconnection voltage of >50 kV during the reconnection burst but a contribution to the convection voltage of >15 kV. However, from the motion of the event we infer that the luminosity peaks at an altitude in the range of 400 and 500 km, and for the top of this range the reconnection and average convection voltages would be increased to >200 kV and >60 kV, respectively. (These are all minimum estimates because the event extends in longitude beyond the field-of-view of the camera). Hence the higher-emission altitude has a highly significant implication, namely that the reconnection bursts which cause the dayside breakup events could explain most of the voltage placed across the magnetosphere and polar cap by the solar wind flow. Analysis of the plasma density and temperatures during the event on January 9, 1989, predicts the required thermal excitation of significant 630-nm intensities at altitudes of 400-500 km.

Auroral and plasma flow transients at magnetic noon

We present observations of a transient event in the dayside auroral ionosphere at magnetic noon. F-region plasma convection measurements were made by the EISCAT radar, operating in the beamswinging “Polar” experiment mode, and simultaneous observations of the dayside auroral emissions were made by optical meridian-scanning photometers and all-sky TV cameras at Ny Ålesund, Spitzbergen. The data were recorded on 9 January 1989, and a sequence of bursts of flow, with associated transient aurora, were observed between 08:45 and 11:00 U.T. In this paper we concentrate on an event around 09:05 U.T. because that is very close to local magnetic noon. The optical data show a transient intensification and widening (in latitude) of the cusp/cleft region, as seen in red line auroral emissions. Over an interval of about 10 min, the band of 630 nm aurora widened from about 1.5° of invariant latitude to over 5° and returned to its original width. Embedded within the widening band of 630 nm emissions were two intense, active 557.7 nm arc fragments with rays which persisted for about 2 min each. The flow data before and after the optical transient show eastward flows, with speeds increasing markedly with latitude across the band of 630 nm aurora. Strong, apparently westward, flows appeared inside the band while it was widening, but these rotated round to eastward, through northward, as the band shrunk to its original width. The observed ion temperatures verify that the flow speeds during the transient were, to a large extent, as derived using the beamswinging technique; but they also show that the flow increase initially occurred in the western azimuth only. This spatial gradient in the flow introduces ambiguity in the direction of these initial flows and they could have been north-eastward rather than westward. However, the westward direction derived by the beamswinging is consistent with the motion of the colocated and coincident active 557.7 nm arc fragment, A more stable transient 557.7 nm aurora was found close to the shear between the inferred westward flows and the persisting eastward flows to the North. Throughout the transient, northward flow was observed across the equatorward boundary of the 630 nm aurora. Interpretation of the data is made difficult by lack of IMF data, problems in distinguishing the cusp and cleft aurora and uncertainty over which field lines are open and which are closed. However, at magnetic noon there is a 50% probability that we were observing the cusp, in which case from its southerly location we infer that the IMF was southward and many features are suggestive of time-varying reconnection at a single X-line on the dayside magnetopause. This IMF orientation is also consistent with the polar rain precipitation observed simultaneously by the DMSP-F9 satellite in the southern polar cap. There is also a 25% chance that we were observing the cleft (or the mantle poleward of the cleft). In this case we infer that the IMF was northward and the transient is well explained by reconnection which is not only transient in time but occurs at various sites located randomly on the dayside magnetopause (i.e. patchy in space). Lastly, there is a 25% chance that we were observing the cusp poleward of the cleft, in which case we infer that IMF Bz was near zero and the transient is explained by a mixture of the previous two interpretations.

Analysis of incoherent scatter radar data from non-thermal F-region plasma

A procedure is presented for fitting incoherent scatter radar data from non-thermal F-region ionospheric plasma, using theoretical spectra previously predicted. It is found that values of the shape distortion factor D∗, associated with deviations of the ion velocity distribution from a Maxwellian distribution, and ion temperatures can be deduced (the results being independent of the path of iteration) if the angle between the line-of-sight and the geomagnetic field is larger than about 15–20°. The procedure can be used with one or both of two sets of assumptions. These concern the validity of the adopted model for the line-of-sight ion velocity distribution in the one case or for the full three-dimensional ion velocity distribution function in the other. The distribution function employed was developed to describe the line-of-sight velocity distribution for large aspect angles, but both experimental data and Monte Carlo simulations indicate that the form of the field-perpendicular distribution can also describe the distribution at more general aspect angles. The assumption of this form for the line-of-sight velocity distribution at a general aspect angle enables rigorous derivation of values of the one-dimensional, line-of-sight ion temperature. With some additional assumptions (principally that the field-parallel distribution is always Maxwellian and there is a simple relationship between the ion temperature anisotropy and the distortion of the field-perpendicular distribution from a Maxwellian), fits to data for large aspect angles enable determination of line-of-sight temperatures at all aspect angles and hence, of the average ion temperature and the ion temperature anisotropy. For small aspect angles, the analysis is restricted to the determination of the line-of-sight ion temperature because the theoretical spectrum is insensitive to non-thermal effects when the plasma is viewed along directions almost parallel to the magnetic field. This limitation is expected to apply to any realistic model of the ion velocity distribution function and its consequences are discussed. Fit strategies which allow for mixed ion composition are also considered. Examples of fits to data from various EISCAT observing programmes are presented.

Radar observations of non-thermal plasmas at different aspect angles

Data are presented from the EISCAT CP-3-E experiment which show the presence of non-thermal plasma over a range of latitudes. The O+ ion-velocity distribution function is almost toroidal when the electric field reaches values of 125 mV m−1. The ion temperature derived from such data assuming a Maxwellian distribution function will overestimate the true ion temperature when the observing angle is large with respect to the magnetic field, and underestimate the temperature when the aspect angle is small. When the expressions for the distribution function are extended to include mixed ion composition, an improvement is sometimes found in fitting the observed data, and estimates of the composition can be made. Such an analysis suggests that N2+ can occasionally form a significant part of the total ion density in a narrow height region centred at 275 km.

Ion flows and heating at a contracting polar-cap boundary: GISMOS observations indicating viscous-like interaction on the flanks of the magnetotail

This paper complements that in this issue by Clauer et al. concerning the international GISMOS campaign of 3–5 June 1987. From a detailed study of the EISCAT data, the polar-cap boundary, as defined by an almost shear east-west convection reversal, is found to contract across the EISCAT field of view between 04 and 07 MLT. An annulus of enhanced ion temperature and non-thermal plasma is observed immediately equatorward of the contracting boundary due to the lag in the response of the neutral-wind pattern to the change in ion flows. The ion flow inside the polar cap and at the boundary is shown to be relatively smooth, compared with that in the auroral oval, at 15-second resolution. The flow at the boundary is directed poleward, with velocities which exceed that of the boundary itself. The effect of velocity shears on the beamswinging technique used to derive the ion flows has been analysed in detail and it is found that spurious flows across a moving boundary can be generated. However, these are much smaller than the observed flows into the polar cap and cannot explain the 7 kV potential difference across the observed segment of the cap boundary between 04:30–06:30 UT. The ion temperature enhancements at the two observing azimuths is used to define the boundary orientation. The results are consistent with recent observations of slow anti-sunward flow of closed field lines on the flanks of the geomagnetic tail, which appears to be generated by some form of “viscous” coupling to the magnetosheath plasma.

Analysis of incoherent scatter spectra from non-Maxwellian plasma

Incoherent scatter data from non-thermal F-region ionospheric plasma are analysed, using theoretical spectra predicted by Raman et al. It is found that values of the semi-empirical drift parameter D∗, associated with deviations of the ion velocity distribution from a Maxwellian, and the plasma temperatures can be rigorously deduced (the results being independent of the path of iteration) if the angle between the line-of-sight and the geomagnetic field is larger than about 15–20 degrees. For small aspect angles, the deduced value of the average (or 3-D) ion temperature remains ambiguous and the analysis is restricted to the determination of the line-of-sight temperature because the theoretical spectrum is insensitive to non-thermal effects when the plasma is viewed along directions almost parallel to the magnetic field. This limitation is expected to apply to any realistic model of the ion velocity distribution, and its consequences are discussed. Fit strategies which allow for mixed ion composition are also considered. Examples of fits to data from various EISCAT observing programmes are presented.

Observations of nonthermal plasmas at different aspect angles

Data are presented from the EISCAT (European Incoherent Scatter (Facility)) CP-3-E experiment which show large increases in the auroral zone convection velocities (>2 km s−1) over a wide range of latitudes. These are larger than the estimated neutral thermal speed and allow a study of the plasma in a nonthermal state over a range of observing angles. Spectra are presented which show a well-defined central peak, consistent with an ion velocity distribution function which significantly departs from a Maxwellian form. As the aspect angle decreases, the central peak becomes less obvious. Simulated spectra, derived using theoretical expressions for the O+ ion velocity distribution function based on the generalized relaxation collision model, are compared with the observations and show good first-order, qualitative agreement. It is shown that ion temperatures derived from the observations, with the assumption of a Maxwellian distribution function, are an overestimate of the true ion temperature at large aspect angles and an underestimate at low aspect angles. The theoretical distribution functions have been included in the “standard” incoherent scatter radar analysis procedure, and attempts have been made to derive realistic ionospheric parameters from nonthermal plasma observations. If the expressions for the distribution function are extended to include mixed ion composition, a significant improvement is found in fitting some of the observed spectra, and estimates of the ion composition can be made. The non-Maxwellian analysis of the data revealed that the spectral shape distortion parameter, D*, was significantly higher in this case for molecular ions than for atomic ions in a thin height slab roughly 40 km thick. This would seem unlikely if the main molecular ions present were NO+. We therefore suggest that N2+ formed a significant proportion of the molecular ions present during these observations.

Non-thermal plasma observations using EISCAT: Aspect angle dependence

Recent observations with the EISCAT incoherent scatter radar have shown large rises in dayside, auroral plasma velocities (>2 km s^{−1}) over a wide range of latitudes and lasting about an hour. These are larger than the neutral thermal speed, and allow, for the first time, observations of a non-thermal plasma over a range of observing angles, revealing a clear angular dependence. The observed ion temperature anisotropy, deduced by assuming a Maxwellian line-of-sight ion velocity distribution, is at least 1.75, which exceeds the theoretical value for a bi-Maxwellian based on a realistic ion-neutral collision model. The aspect angle dependence of the signal spectra also indicates non-Maxwellian plasma.

Vertical structure and physical processes of the Madden-Julian oscillation: Biases and uncertainties at short range

An analysis of diabatic heating and moistening processes from 12-36 hour lead time forecasts from 12 Global Circulation Models are presented as part of the "Vertical structure and physical processes of the Madden-Julian Oscillation (MJO)" project. A lead time of 12-36 hours is chosen to constrain the large scale dynamics and thermodynamics to be close to observations while avoiding being too close to the initial spin-up for the models as they adjust to being driven from the YOTC analysis. A comparison of the vertical velocity and rainfall with the observations and YOTC analysis suggests that the phases of convection associated with the MJO are constrained in most models at this lead time although the rainfall in the suppressed phase is typically overestimated. Although the large scale dynamics is reasonably constrained, moistening and heating profiles have large inter-model spread. In particular, there are large spreads in convective heating and moistening at mid-levels during the transition to active convection. Radiative heating and cloud parameters have the largest relative spread across models at upper levels during the active phase. A detailed analysis of time step behaviour shows that some models show strong intermittency in rainfall and differences in the precipitation and dynamics relationship between models. The wealth of model outputs archived during this project is a very valuable resource for model developers beyond the study of the MJO. In addition, the findings of this study can inform the design of process model experiments, and inform the priorities for field experiments and future observing systems.

Methane cross-validation between three Fourier Transform Spectrometers: SCISAT ACE-FTS, GOSAT TANSO-FTS, and ground-based FTS measurements in the Canadian high Arctic

We present cross-validation of remote sensing measurements of methane profiles in the Canadian high Arctic. Accurate and precise measurements of methane are essential to understand quantitatively its role in the climate system and in global change. Here, we show a cross-validation between three datasets: two from spaceborne instruments and one from a ground-based instrument. All are Fourier Transform Spectrometers (FTSs). We consider the Canadian SCISAT Atmospheric Chemistry Experiment (ACE)-FTS, a solar occultation infrared spectrometer operating since 2004, and the thermal infrared band of the Japanese Greenhouse Gases Observing Satellite (GOSAT) Thermal And Near infrared Sensor for carbon Observation (TANSO)-FTS, a nadir/off-nadir scanning FTS instrument operating at solar and terrestrial infrared wavelengths, since 2009. The ground-based instrument is a Bruker 125HR Fourier Transform Infrared (FTIR) spectrometer, measuring mid-infrared solar absorption spectra at the Polar Environment Atmospheric Research Laboratory (PEARL) Ridge Lab at Eureka, Nunavut (80° N, 86° W) since 2006. For each pair of instruments, measurements are collocated within 500 km and 24 h. An additional criterion based on potential vorticity values was found not to significantly affect differences between measurements. Profiles are regridded to a common vertical grid for each comparison set. To account for differing vertical resolutions, ACE-FTS measurements are smoothed to the resolution of either PEARL-FTS or TANSO-FTS, and PEARL-FTS measurements are smoothed to the TANSO-FTS resolution. Differences for each pair are examined in terms of profile and partial columns. During the period considered, the number of collocations for each pair is large enough to obtain a good sample size (from several hundred to tens of thousands depending on pair and configuration). Considering full profiles, the degrees of freedom for signal (DOFS) are between 0.2 and 0.7 for TANSO-FTS and between 1.5 and 3 for PEARL-FTS, while ACE-FTS has considerably more information (roughly 1° of freedom per altitude level). We take partial columns between roughly 5 and 30 km for the ACE-FTS–PEARL-FTS comparison, and between 5 and 10 km for the other pairs. The DOFS for the partial columns are between 1.2 and 2 for PEARL-FTS collocated with ACE-FTS, between 0.1 and 0.5 for PEARL-FTS collocated with TANSO-FTS or for TANSO-FTS collocated with either other instrument, while ACE-FTS has much higher information content. For all pairs, the partial column differences are within ± 3 × 1022 molecules cm−2. Expressed as median ± median absolute deviation (expressed in absolute or relative terms), these differences are 0.11 ± 9.60 × 10^20 molecules cm−2 (0.012 ± 1.018 %) for TANSO-FTS–PEARL-FTS, −2.6 ± 2.6 × 10^21 molecules cm−2 (−1.6 ± 1.6 %) for ACE-FTS–PEARL-FTS, and 7.4 ± 6.0 × 10^20 molecules cm−2 (0.78 ± 0.64 %) for TANSO-FTS–ACE-FTS. The differences for ACE-FTS–PEARL-FTS and TANSO-FTS–PEARL-FTS partial columns decrease significantly as a function of PEARL partial columns, whereas the range of partial column values for TANSO-FTS–ACE-FTS collocations is too small to draw any conclusion on its dependence on ACE-FTS partial columns.