Evaluation of the Forecast Accuracy of Near-Surface Temperature and Wind in Northwest China Based on the WRF Model

Numerical model assessment is carried out to fully integrate the available observational data, to analyze the capability/deficiency of the model, to refine the relevant content of the model based on understanding of the key issues involved, and finally to improve the model’s ability in simulating and predicting certain phenomena. However, how to correctly understand numerical forecast products and identify model problems and shortcomings remains a formidable task, but one that is important for improving forecast results. Inspection of various types of forecast products in different countries has become an effective way to improve the accuracy of local weather forecasting. Testing and understanding of various error characteristics and sources not only help forecasters correctly explain the forecast products, but also help developers understand the numerical model performance, thus pro-viding a base for further model improvement.

Forecasts of near-surface variables, such as temperature at 2-m height (i.e., 2-m temperature) and winds at 10-m height (i.e., 10-m winds), are essential tasks. These two variables are mostly concerned in daily weather forecasts and services. Previous studies have indicated that forecasts of near-surface variables using numerical models are closely related to the specification of the underlying surface state, including topography, surface heat and energy transfer, radiative processes, land surface process, and boundary layer processes (Lee et al., 1989; Cheng and Steenburgh, 2005; Ruiz et al., 2010; He et al., 2014; Pei et al., 2014; Siuta et al., 2017). However, the understanding of these processes is limited, and there is considerable uncertainty in the parameterization of these physical processes. Moreover, the near-surface atmos-phere interacts directly with the ground surface, and because of the dynamic and thermodynamic exchanges be-tween the surface and atmosphere in the boundary layer, the near-surface atmosphere exhibits unique characteris-tics (Stull, 1988). For example, there is diurnal variation in near-surface temperature and wind fields, which is different from the free atmosphere. The aforementioned aspects make it difficult to predict near-surface variables using models (Hanna and Yang, 2001; Zhang and Zheng, 2004). Therefore, a complete evaluation of model simulations of near-surface variables is necessary and important to model development.

Researchers in China and other countries have carried out a number of studies on the evaluation of forecasts of surface variables. Zhong and Fast (2003) compared three mesoscale models and evaluated their simulations over the Salt Lake valley under weak and strong weather scenarios. They found that there is cold bias in the valley extending from the surface to the top of the atmosphere. The nighttime 2-m temperature simulated by the model is much weaker than the observed. Even under strong weather forcing, there is significant error in the wind forecast. He et al. (2014) noted that near-surface temperatures are very sensitive to the resolution of surface data; the influence of surface conditions on temperature throughout the boundary layer is vital for accurate surface information and for improving the simulation of near-surface and boundary layers in the Weather Research and Forecasting (WRF) model. Santos-Alamillos et al. (2013) evaluated the relative role of each parameterization scheme and terrain representation using the WRF model for the assessment of regional wind power resources. They noted that the terrain and the forms of terrain have a significant impact on simulation of surface meteorological variables, especially surface wind speed and wind direction.

Generally speaking, the simulated near-surface wind speed in the afternoon from a mesoscale model is often relatively low (Zhang and Zheng, 2004), while the simulated wind speed at night is larger (Mass et al., 2002; Byun et al., 2008; Lee et al., 2011; Chen et al., 2012; Ngan et al., 2012, 2013; Hu et al., 2013). Byun et al. (2008) noted that the higher forecasted nighttime wind speed is often related to clear weather dominated by southwest-southerly wind. Zhang and Zheng (2004) indicated that when there is a low-level jet near the top of the nighttime boundary layer, the downward transfer of momen-tum could cause higher forecasted wind speed. The simulated low-level wind in the afternoon is more accurate than that in early morning, which is usually unstable (Wang and Jin, 2013). Lee et al. (2011) indicated that larger forecast values for sensible heat flux and nighttime boundary layer height make vertical mixing stronger, and the enhanced downward transfer of momentum causes higher forecasted wind speed. Zhang et al. (2013) found that deviation in forecast of near-surface variables over flat terrain depends largely on atmospheric static stability.

Through many observational and model simulation studies, it is confirmed that terrain exerts important impacts on atmospheric circulation, rainfall, and near-surface variables (Smith and Lin 1982; Wu and Schlunzen, 1992; Rowell and Milford, 1993; Sheng et al., 2000; Jin et al., 2016). Terrain blocks airflow, generating disturbed motion; and alteration of the wind field directly affects the transport of momentum, heat, and moisture. These further affect the variation of meteorological variables (e.g., rainfall, temperature, and wind speed) that are of most concern in weather forecasts (Tao, 1980). Whiteman (2000) showed that topography produces strong modifications in synoptic-scale circulation, leading to high spatial variability in surface flow over areas of complex terrain. Jiménez and Dudhia (2012) showed that deviation of a 10-m wind forecast by the Advanced Research WRF model relied on the type of terrain, and they highlighted the important drawbacks of predicting wind over complex terrain. Jiménez and Dudhia (2013) found differences between WRF-simulated and in situ surface wind direction, which are greater in magnitude over areas of complex terrain compared with flatter terrain. WRF terrain representation may provide unreliable surface wind estimates in areas of complex topography, even with high spatial resolution (Rife et al., 2004; Santos-Alamillos et al., 2013). Data assimilation is more difficult in complex terrain than in plain terrain (Mesinger et al., 2006; Ancell et al., 2011; Pu et al., 2013; Gómez-Navarro et al., 2015). A comparison of four mesoscale models by Hanna et al. (2001) indicated that uncertainty in the simulation of near-surface wind speed and direction is caused mainly by random or turbulent disturbance. In regions with complex terrain, the forecast error of wind speed increases because data input to the model are limited. Meanwhile, sub-grid variation of terrain and land-use will also cause this variation. Therefore, the forecast of surface variables in areas with complex terrain is more difficult, and the validation and evaluation of surface variables in the model appear to be more important in such cases. However, most previous studies focused on the verification of large-, synoptic-, and mesoscale cases and neglected the exclusive assessment of near-surface atmospheric conditions.

Diurnal variations in temperature and wind are influenced by the interaction between ground surface and atmosphere due to differences in land use/land cover. Numerous studies have confirmed the influence of changes in the underlying surface on regional climate (Yucel, 2006; Pan et al., 2012; Chen and Zhang, 2013; Cao et al., 2015; Chen et al., 2015; Lorente-Plazas et al., 2016). Furthermore, land-use changes across China have experienced significant spatial and temporal variations in the past 20 years (Liu et al., 2014). Many studies have investigated the feedback relationship between vegetation cover/land use and precipitation (Kim and Wang, 2007; Lim et al., 2008; Comarazamy et al., 2013; Meng et al., 2014). The influence of vegetation on rainfall changes with the timing and direction of the initial water content anomaly in the soil. In June, the moisture anomaly in the soil of wetlands enhances the leaf area index and causes increased evapotranspiration and surface heating, further increasing rainfall. Scheitlin and Dixon (2010) investigated the relationship between the temperature difference between daytime and nighttime in southeastern United States and the land use/land cover (cities, agriculture, evergreen forest, deciduous forest, mixed forest, etc.); they noted that the temperature difference between daytime and nighttime is smallest in farmland and cities and largest in the forests. The influence of vegetation cover can extend upward to the free atmosphere and further affects the atmospheric circulation (Kabat et al., 2004). Hirsch et al. (2015) investigated the relationship between land-use changes, land–atmosphere coupling, and extr-eme temperature during summer in Australia, and their results indicated that the influence of land-use changes on extreme temperature depends on the selection of the planetary boundary layer (PBL) scheme. This emphasizes that different combinations of convection and PBL schemes affect how land use/land cover impacts the WRF model results. The wind at lower levels is readily influenced by the underlying surface, and the model parameterization scheme is imperfect near the surface (Zhang et al., 2015). Surface winds result from the interaction between mesoscale circulation and other more lo-cal variables, many of which are related to topographic characteristics (elevation, aspect, and slope) and terrain morphology (hills, valleys, etc.) (Santos-Alamillos et al., 2013).

Similar to the aforementioned studies in China and other countries, validation and evaluation of surface variables’ forecasts have also been conducted in the United States, but mostly limited to the regions that have the potential for severe weather. The regions that have been validated are mostly local, small-range regions. The validation studies in China have been carried out mostly in southern and eastern China. There have not been many efforts to validate and assess forecasts of surface variables in Northwest China, especially in this region with complex terrain. The current study intends to carry out such an assessment in Northwest China, where the terrain, surface vegetation, and soil types are varied, and complex mountain ranges intersect, with greatly varying altitudes. Moreover, Northwest China is affected by the interaction of the East Asian summer monsoon, the Tibetan Plateau weather system, and the westerly belt weather system. Dynamic and thermal effects co-exist and cause diverse high-impact weathers. Therefore, it is important to test and verify model forecasts of surface variables in this area. At present, there are few studies that have done such validation and assessments in Northwest China.

In this paper, the capability of the WRF model in forecasting surface variables in Northwest China in summer and winter 2015, especially the 2-m temperature and 10-m wind speed and direction, is evaluated first. The forecast capability of the WRF model in different seasons is then analyzed, along with characteristics of the forecast errors for surface variables under complex underlying surface states. In addition, the model prediction error and its diurnal variation under different weather conditions are also compared. We mainly evaluate the forecasts in June and December 2015, and reveal the characteristics of the continuous forecasting errors of near-surface variables. The structure of the paper is as follows. Section 2 briefly introduces the model system and its design scheme. Section 3 describes the method of validation, and Section 4 discusses the results, including the characteristics of errors for different variables in different simulation regions and for different terrain heights and land-use types. Section 5 presents a summary and discussion.

The model used in this study is WRF version 3.6.1, with 40 vertical layers and the Lambert map projection. The model domain is centered at 39°N, 100°E. The forecast area of the model is shown in Fig. 1, where domains 1 and 2 (D01 and D02) are two-way nested but domain 3 (D03) adopts one-way nesting. The initial and boundary conditions of the model are from the final analytical fields of the US NCEP Global Forecast System (GFS) (https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs); the terrain data with a 30-arc-s resolution are downloaded at http://www2.mmm.ucar.edu/wrf/users/download/get_sources_wps_geog.html); and the 24 classes of land-use data are also downloaded at the above website with a different file name. The topography and land-use data are both developed by the US Geological Survey (USGS). The horizontal resolution, model domains, and the physical schemes selected in each domain are given in Table 1.

Figure 1.  Simulation domains and distribution of observation stations. Note that the Lambert map projection is used here, which is different from that in other figures in this paper.

June and December 2015 are selected for comparative analysis. We use 2000 Beijing Time (BT) as the initial time of forecast for the daily 48-h prediction using WRF. Because the GFS coarse-grid analytic fields are used, the first 12 h is considered as spin-up time, only the daily prediction results from 0800–0800 BT (next day) are compared with observations to validate the WRF forecast results. For a fair comparison, bilinear interpolation is applied to interpolate the forecast results of different variables to the observation stations.

The distribution of ground stations used for the validation is shown in Fig. 1. There are 424 stations in D03. Because D01 and D02 are two-way nested, we only evaluate the results of 9-km resolution in D02. We also evaluate the results in D03 on a 3-km resolution (Table 1). The observation data used in this paper are the quality-controlled ground observations released by the China Meteorological Administration, for which the reliability is ensured.

Prediction errors of near-surface variables often exist due to different terrain heights between the model and the reality. Therefore, model terrain heights are compared to actual terrain heights to check for representation errors. Then, the variation of the model prediction error at different terrain heights is evaluated. Variables including 2-m temperature and 10-m wind speed and direction are validated by calculating bias error (BE), absolute error (AE), mean bias error (MBE, regional average), and mean absolute error (MAE, regional average). We also calculate time-averaged mean bias error (TMBE) and time-averaged mean absolute error (TMAE) over June and December. Because the observation error of wind direction is usually larger at lower wind speeds, the wind direction used for validation in this paper are those with speeds greater than 5 m s^–1. The formulas to calculate the different statistical quantities are from Zhang et al. (2013).

Figure 2 shows horizontal distributions of the monthly average TMBE of 2-m temperature. It is apparent that the BE distribution in D02 and D03 has unified features in summer; namely, the BE distribution of 2-m temperature is associated with the terrain distribution. In regions with relatively strong radiation and evaporation, such as the Tarim basin, Qaidam basin, Hexi Gobi of Gansu Province, Badain Jaran Desert, Tengger Desert, Mu Us Sandland, Hunshandake Sandland, and Guanzhong basin, the forecasted temperature is higher, and the BE is approximately 2°C. The BEs of temperature in D02 and D03 in June are smaller than those in December, and the error is approximately –2°C. In addition to the presence of seasonal difference in surface albedo that affects radiation, the freezing–thawing process of soil during seasonal transition in Northwest China is not fully considered in the land-surface schemes in WRF. In December, the soil in Northwest China gradually begins to freeze, releasing a large amount of heat that prevents the near-surface temperature from declining to a very low value. However, the Noah land-surface model (LSM) used in WRF somewhat overestimates the heat flux and soil temperature, which might be associated with the energy closure problem against eddy covariance observations (Zheng et al., 2014); the Noah LSM overestimates the daytime turbulence and latent heat flux. But the temperature in frozen soil (December–February) is underestimated, resulting in the forecast of near-surface air temperature lower than the observations (Zheng et al., 2017a, b). Another possible reason is that these regions are dominated by the Gobi Desert, and there is a clear radiation influence of dust aerosols in June. Because the dust aerosols at mid to high levels absorbs radiation, the radiation reaching the ground surface is relatively less intense, and the temperature at the ground surface is lower (Xin et al., 2016). However, the WRF model does not thoroughly consider the factor of dust aerosols, which makes the radiation reaching the ground surface in the model higher than the actual situation, and the temperature correspondingly higher. Zhang et al. (2010) demonstrated that aerosols can reduce incoming solar radiation by up to –9% in January and –16% in July and 2-m temperatures by up to 0.16°C in January and 0.37°C in July over most of the continental U.S; in December, however, there is no influence of dust aerosols, and the forecast error of temperature is smaller. These, however, cannot fully explain the BE features shown in Fig. 2 in our case.

Figure 2.  Horizontal distributions of monthly TMBE for 2-m temperature: (a, c) average in June and (b, d) average in December over (a, b) D02 and (c, d) D03. Background shading in black and white denotes terrain height (m).

As Fig. 2b indicates, the errors in the Guanzhong basin，North China Plain, and Henan and Hubei provinces, which have relatively low topography, are positive in June and December. The forecasted values in other regions of China are lower than observed values, and the cold bias (negative error) is more apparent in the plateau area, especially to the east of the plateau, eastern Tibet, southwest of Sichuan Province, and north of Yunnan Province. In addition, in some regions of high terrain, such as the Tianshan Mountains, Qilian Mountains, Dabashan Mountains, and Lüliang Mountains, there is a clear cold bias in the forecast, and the error is approximately –2°C. The error is particularly larger in southeastern Tibetan Plateau (TP). One possible reason is that there are relatively few measurement stations in southeastern TP, and the terrain gradient is bigger. The bias between the model terrain height and the station height and the inaccurate description of land types in the model cause larger temperature errors. A comparison of the distribution of BE for temperatures in D02 and D03 indicates that the BE in D03 for the temperature forecast is significantly improved over that in D02 in June and December (Figs. 2c, d), with much reduced cold biases over the Qilian Mountains (northeast of TP) and less warm biases east of the Tibetan Plateau.

The above analysis indicates that terrain height has an important influence on temperature and wind field prediction. Therefore, the impact of different terrain heights on the prediction of near-surface features will be discussed in Section 4.3.

The distribution of TMBE for 10-m wind speed is shown in Fig. 3. The forecasted wind speed is generally greater than observations over the entire domain in June. The model behaves better during winter than in summer. This may be because local and mesoscale processes are more relevant and difficult to represent in summer, while there are always large-scale processes that dominate in winter (Wang and Jin, 2013). In particular, in the desert zones of the Hexi Corridor, Qaidam basin, Badain Jaran Desert, Tengger Desert, and Mu Us Sandland, the forecasted wind speed is greater, and BE is smaller in December than in June. The regions with large BEs are the same as those in June, with exceptions in Qijiaojing of Xinjiang and Guaizihu of Inner Mongolia, where the forecasted wind speeds are less than the actual values. In particular, in Qijiaojing, on the southern side of the Maertala Mountains in eastern Tianshan Mountains, it is dry throughout the year and evaporation is very high; it is the famous “100-km wind zone” in Xinjiang. However, Guaizihu is on the northern border of the Badain Jaran Desert in Ejinaqi of Inner Mongolia. In the 1920s and 1930s, it was a mosaic of grassland and shrubland. Under the impact of global climate change and unsustainable human activities, the underground water level of the Badain Jaran Desert supplying Guaizihu has continued to decline. By the end of 2009, Guaizihu was completely dry, the area of the oasis had shrunk significantly, and desertification was accelerated. It has now become an area of desert and sparse vegetation, and there has been a tremendous change in land use, which could be the main reason for the lower forecasted wind speeds.

Figure 3.  Horizontal distributions of monthly TMBE for 10-m wind speed: (a, c) average in June and (b, d) average in December over (a, b) D02 and (c, d) D03. Background shading in black and white denotes terrain height (m).

A comparison of the BEs of wind speed in D02 and D03 shows that the BE in D03 is obviously lower than that in D02 in both June and December, which indicates that better resolution can improve wind speed forecasts.

Figures 4a and 4b show the daily evolution of regional averaged bias error (MBE) for temperature in D02 and D03 in June and December 2015. It is apparent that the forecast trend and characteristics of diurnal variation of temperature are consistent over the two domains. MBEs of temperature in the two domains are within ±2°C (±4°C) in June (December) except for a few days, and are generally a bit larger in D03 than in D02.

Figure 4.  Daily evolution of MBE for different variables in D02 (solid line) and D03 (dashed line): (a, b) temperature; (c, d) wind speed; and (e, f) wind direction in (a, c, e) June and (b, d, f) December 2015.

From the evolution of MBE for wind speed (Figs. 4c, d), it is inferred that the wind speed forecasted by the model in June is generally higher than that in December. The average MBE is 1–2 m s^–1, and the largest MBE is 4 m s^–1. A comparison of MBE in D02 and D03 shows that MBE in D03 is smaller than that in D02. Compared to June, variation in MBE of wind speed is larger in December. From 1 to 3 December, the error of wind speed is larger; from 3 to 11 December, the error is smaller, and the average is 1 m s^–1; during 12–15 December, the error increases again, and the average is approximately 2 m s^–1; during 16–31 December, the error variation is relatively stable, and the average is 1 m s^–1. During the entire process, the error variation is consistent in D02 and D03. Based on a general survey of weather processes in December 2015, a gale occurred in the Hexi Corridor of Gansu Province from 1 to 3 December, and rainfall occurred in Northwest China during 10–15 and 24–26 December. When there is a weather process, the declining trend in temperature predicted by the model is smaller than the observation, and the simulated temperature is also greater than the observation. Due to the thermodynamic effect, simulated wind speed is also greater than the observed. Generally speaking, the forecast trend and evolution attributes of wind speed are consistent in different domains of the simulation. In particular, the forecasted wind speed in D03 is closer to the observed value.

The daily evolution of MBE in wind direction demonstrates that wind direction predictions in D02 and D03 are not considerably different during June and December. Since the error of wind direction used for evaluation is from wind directions with speeds greater than 5 m s^–1, and the wind direction is the average of all stations used for evaluation, this causes a small difference in wind direction errors between D02 and D03. However, under certain weather processes, high resolution still improves the wind direction forecasts.

Figure 5 shows diurnal variations of observations and forecasts for temperature, wind speed, and wind direction. It is apparent that the WRF model has a good ability to forecast diurnal variation in temperature. The forecasted values in D02 and D03 are slightly higher than the observed in June; in particular, the value in D02 is closer to the observed, and that in D03 is slightly higher than that of D02 by approximately 1°C (Fig. 5a). In December, the 2-m temperature predictions in D02 and D03 are lower than observations during daytime, but a little higher than the observed at night (Fig. 5b). That is, the model weakens the diurnal variation of temperature. In particular, the forecasted value in D03 is generally somewhat higher than that in D02, but closer to the observed value. The BE is approximately 2°C. The diurnal cycle of the predicted 2-m temperature depends on many factors, but these can be aggregated into two major components: surface energy balance, which depends on insolation, land surface properties, and atmospheric states; and surface thermal inertia, which depends on the type of soil, its moisture content, and vegetation cover (Price, 1977, 1980; Oke, 1987). One possible reason for the lower forecast of diurnal 2-m temperature in the model is that WRF overestimates the mean shortwave downward total solar radiation flux, which seems to be related to a very low cumulus cloud amount in the model and, possibly, a misrepresentation of the radiative impact of this type of cloud.

Figure 5.  Diurnal variations of (a, b) temperature; (c, d) wind speed; and (e, f) wind direction averaged over D02 and D03 in comparison with observations in (a, c, e) June and (c, d, f) December 2015.

For diurnal variation in wind speed (Figs. 5c, d), there is also a clear diurnal variation in June and December. Namely, wind speed is higher in the daytime and smaller at night. In June, the forecasted wind speeds in D02 and D03 are close but are generally higher than observation, and the error is approximately 2 m s^–1. However, the forecasted diurnal variation in the wind speed is consistent with observation. In December, the forecast error of wind speed in D03 is smaller than that in D02, and closer to the observed value. In short, there is a clear overestimate of wind speed in both June and December, similar to what has been shown in other studies around the world. Cheng and Steenburgh (2005) found that the WRF model has presented a high surface wind speed bias over land since its early versions. Jiménez and Dudhia (2012) argued that a plausible explanation for the high bias could be the smooth topography used in the model to simulate atmospheric evolution. Not taking into account the additional resistance effects of vegetation on the unresolved terrain, as in the case of WRF, may lead to an overestimation of wind speed.

The forecasted wind direction also shows a certain diurnal variation (Figs. 5e, f). In June, the winds are mainly southeasterly from midnight to noon, and turn into southerly in the afternoon in Northwest China, but they are slightly deflected. However, the simulated wind direction is from southwest to south during midnight to afternoon, and from south to southeast during afternoon to midnight. Unlike in June, the southwesterly winds dominate during day and night in December, while the model forecasts are mostly south-easterly winds. Jiménez and Dudhia (2013) pointed out that differences between WRF and in situ surface wind directions are larger in magnitude over areas of complex terrain than in areas of flat terrain. Our results show that systematic differences are found over complex terrain in Northwest China, even at high wind speeds and high horizontal resolution.

Wind direction is measured instantaneously, so we use temporal/regional average to eliminate its instantaneous feature. Here, Minqin and Pingliang stations are selected to represent plain and complex terrain, respectively. Table 2 reveals that the average wind speed (SpdAve) at Minqin is 3 m s^–1, and the standard deviation (SpdStd) is 2 m s^–1. The average wind is northwest, and there are some differences in the wind direction forecasted in the two domains. D02 is controlled by east-southeast wind, and D03 is controlled by northeast wind, so the predicted wind direction in D03 is closer to the observation (Fig. 6). The forecasted wind speeds in these two domains are slightly higher, by 2 m s^–1. The forecasted average wind speed at Pingliang is 2 m s^–1, greater than the observed speed; and the average wind direction (DirAve) is north. The forecasts in the two domains are both east-southeast wind, but the forecast error of D03 is smaller.

Figure 6.  Comparison of wind speed and direction between (a, d) observations and (b, c, e, f) forecasts at (a, b, c) Minqin and (d, e, f) Pingliang stations in (b, e) D02 and (c, f) D03. The orientation of the lines represents wind direction and different colors represent the magnitudes of wind speed. The length of the lines is indicated by numbers, denoting the probability.

Terrain plays a vital role in the simulation and forecasting of rainfall and temperature. Figure 7 compares the actual and model terrain heights at the observation stations. The terrain height of the model is interpolated to the station. From Fig. 7, it is seen that the model terrain height is generally higher than that of the station. When station height is 1500–3500 m, the bias of terrain height between the station and the model is larger. The model terrain height in D03 is closer to that of the station, indicating that the higher the horizontal resolution, the better the model’s description of topography.

Figure 7.  Comparison of terrain height between station data and model data (interpolated to the station locations) in D02 and D03.

Zhang et al. (2013) divided terrain into two types: valleys and mountains. Jiménez and Dudhia (2012) classified terrain into three types: plains, mountains, and complex terrain. However, their focus areas were small, whereas our study area is wide, including all of North-west China. Considering the low representation of above the three categories over a wide range, terrain is classified according to height. The elevations of measurement stations are divided into four categories: < 1000 m, 1000–2000 m, 2000–3000 m, and > 3000 m.

Figure 8 compares TMAE for different variables under the above four elevation ranges. In June, the relative error of 2-m temperature is less than 2°C under all elevations (Fig. 8a). In particular, the errors in D02 and D03 are the largest for < 1000 m, and the error in D03 decreases as the elevation increases. Moreover, the error in D03 is close to 0 when the elevation is > 3000 m, which is likely because there are few stations above > 3000 m (only 25) in D03, and there are fewer samples in D03 than in D02. A comparison of the relative error for the forecast of wind speed in D02 and D03 indicates that the forecasted wind speeds in the two domains are not considerably different. Except for the areas with elevations between 1000 and 2000 m, where the error is larger, the relative error for other elevations decreases as the height increases. After monthly averaging of forecasted wind direction, the relative error for different elevations is less than 1.8°C. The forecast error of wind direction is smallest in the areas with elevations between 1000 and 2000 m, followed by < 1000 m, and the error is the largest at > 3000 m. Specific statistics are shown in Table 3.

Figure 8.  TMAE of different variables for different elevations: (a, b) temperature; (c, d) wind speed; and (e, f) wind direction in (a, c, e) June and (b, d, f) December 2015.

TMAE of 2-m temperature in December in D02 and D03 increases as the elevation increases Fig. 8b. A maximum error occurs in the areas with elevations of 2000–3000 m, and the error decreases slightly at > 3000 m. In particular, the forecast error of D02 is greater than that of D03, and the error is the smallest for > 3000 m. The main reason is that the terrain height described by the model in D02 is higher than in D03, especially for areas of 2000–3000 m. Therefore, the vertical rate of temperature decline in D02 is higher than that in D03, causing the forecast error of temperature in D02 to be greater than that in D03. However, a possible reason for the increased forecast error of temperature with increasing height in December is the presence of permafrost and seasonally frozen soil in the northwest plateau area during December. Frozen soil forms more easily at higher elevations. Due to the freezing–thawing process of soil, the forecast temperature at higher elevations is lower in the model than in observations.

In addition, for the forecasted relative errors in the two domains, at elevations < 3000 m, the errors decrease slightly as the elevation increases, and the errors increase slightly at > 3000 m. The variation in the relative error of wind direction is the same as that for wind speed.

Figure 8 compares the 2-m temperature in June and December. When the elevation above sea level is < 2000 m, the relative error is greater in D03 than in D02 in June. As the terrain height increases, the error of forecasted temperature decreases; while the opposite is true in December.

Based on the above analysis, differences in elevation have an important influence on the forecast of surface variables, and the forecast of temperature in different seasons has different responses to elevation. The improvement of wind speed forecast is independent of height, and refined resolution can reduce the wind speed forecast error.

Diurnal variations of the surface variables are clearly different in different seasons and at different elevations (Fig. 9). In June (Fig. 9a), TMAE of 2-m temperature does not have significant diurnal variation at elevations < 1000 m, and TMAE is in general higher than observation. The maximum error is 2.72°C appearing at 0800 BT and the minimum error is 0.44°C appearing at 0500 BT. When the terrain height is 1000–2000 m, the diurnal variation in TMAE of 2-m temperature is clearer than at < 1000 m; the maximum TMAE is 1.67°C, which appears at 0800 BT, and the minimum is –0.39°C, which appears at 1400 BT. When the terrain height is > 2000 m, TMAE of 2-m temperature exhibits an apparent diurnal variation; at 2000–3000 m, the maximum TMAE is 0.82°C at 0800 BT and the minimum is –2.10°C at 2000 BT. When the terrain height is > 3000 m, the maximum TMAE is 0.55°C at 0800 BT and the minimum is –2.90°C at 2000 BT. In December (Fig. 9b), the diurnal variation of the error of 2-m temperature is more obvious than in June. Under different terrain heights, the minimum TMAE of 2-m temperature appears at 1700 BT, while the occurrence time of maximum TMAE is not considerably different, around 0500 or 0800 BT. In addition, in the daytime, as terrain height increases, the absolute value of TMAE of 2-m temperature increases, while at night, it decreases as height increases. Therefore, the WRF simulation weakens the diurnal variation of 2-m temperature, and this weakening is exaggerated as the terrain height increases.

Figure 9.  Diurnal variation in TMAE for different variables at different elevations: (a, b) temperature; (c, d) wind speed; and (e, f) wind direction in (a, c, e) June and (b, d, f) December 2015.

Figure 10.  Types of land use (color shading) and distribution of measurement stations (black dots) in D03.

The forecasted wind speed in the model is higher than that observed during both daytime and nighttime. In June (Fig. 9c), TMAE of 10-m wind speed shows clear diurnal changes with different terrain heights. As the sun rises, TMAE of wind speed gradually increases, reaching a maximum at 1700 BT. Then, it gradually declines and reaches the minimum until 0800 BT. In particular, when terrain height is > 2000 m, the diurnal variation is most significant. The maximum TMAE of wind speed at different heights is not considerably different and is approximately 2 m s^–1, but the minimum TMAE is clearly different, i.e., it is below 0.8 m s^–1 for terrain heights > 2000 m, and it reaches 1.9 m s^–1 for terrain heights of 1000–2000 m. Unlike in June, TMAE of 10-m wind speed does not demonstrate clear diurnal variation in December. The TMAE is smallest at 0800 BT and then gradually increases. It reaches a high value at 1100 BT and then decreases slightly. Until 1700 BT, it declines to the second lowest value and then increases again. After 2000 BT, when it reaches the maximum, the TMAE remains stable, and then gradually declines after 0500 BT the next day until reaching the minimum at 0800 BT.

The prevailing wind direction in Northwest China during June is mainly southwesterly, while it is northwesterly in December. Therefore, when TMAE of wind direction during June is positive, the forecast of westerly wind is larger; if TMAE of wind direction is negative, the predicted southerly wind is larger. In December, because the prevailing wind direction is northwest, when TMAE of wind direction is positive, the forecast of northerly wind is larger; when TMAE of wind direction is negative, the forecast of westerly wind is larger. Figures 9e and 9f show the diurnal variation in TMAE of wind direction in June and December 2015. It is clear that the diurnal variation in TMAE exhibits considerable differences with different terrain heights. In June, when terrain height is < 1000 m, there is a clear diurnal variation in TMAE of wind direction, and the error of wind direction is positive in the daytime. As the sun rises, TMAE of wind direction gradually increases and reaches a maximum of 25.14° at 1700 BT. In the daytime, the forecast of westerly wind is larger, and the forecast of wind direction is mainly westerly. After 2000 BT, TMAE begins to gradually decrease and reaches a minimum of –31.54° at 0200 BT of the next day. During this time, the forecast of southerly wind is higher. After that, the forecasted wind direction is mainly southerly, and TMAE of wind direction gradually increases again; when the terrain height is 1000–2000 m, TMAE of wind direction is within ±10° without obvious diurnal variation. When the terrain height is 2000–3000 m, TMAE of wind direction is negative from 0200 to 0800 BT, and positive at other times. As the sun rises, TMAE gradually increases, which indicates that the westerly component gradually increases until reaching a maximum at 2000 BT; the TMAE is 42.78°, with the forecasted wind direction mainly westerly or west-northwesterly; when the terrain height is > 3000 m, TMAE of wind direction is clearly different than it is at other terrain heights. Except for 1100 BT, the TMAE is negative at other times of the day, which indicates that the wind direction is mainly southerly. In December, except for the heights of 2000–3000 m, when TMAE of wind direction is positive in the daytime and the forecast of northerly wind is larger, TMAE of wind direction is negative within one day at other terrain heights, which indicates that the westerly component forecasted by the model in December is larger than the observed values.

As shown by Cheng et al. (2013), the underlying surface has an important impact on the prediction result of a model. In this paper, all stations contained in D03 are selected and classified according to different types of land use. D03 contains mainly arid and semi-arid areas in Northwest China. The locations of stations and the corresponding types of land use are shown in Fig. 10. The numbers used to represent land-use types in Fig. 10 are illustrated in Table 4. Land use in D03 is dominated mainly by dryland farms, irrigated cropland, cropland, grassland, shrub land, and barren land.

In June, for TMBE of temperature forecasted in D03, except for barren land, the error for all other types of land use is larger than in D02 (Fig. 11a). The BE is smallest for barren land, which is closest to the observed. In D02, BE of temperature is smaller for dry land, farmland, irrigated cropland, farmland/grassland mosaic, and shrub land than in D03, while BE for barren land is larger than in D03. In particular, temperature bias is negative for grassland and barren land and is positive for other types of land use in the simulation. In comparison with TMBE of temperature in December, except for dryland, farms, and irrigated cropland in D02 and D03, where BE of temperature is positive, the TMBE is larger in D03 than in D02. TMBE of the forecasted temperature under other types of land use is negative; moreover, it is smaller in D03.

Figure 11.  TMBE of different surface variables for different types of land use: (a, b) 2-m temperature; (c, d) 10-m wind speed; and (e, f) 10-m wind direction in (a, c, e) June and (b, d, f) December 2015.

A comparison of the characteristics of TMBE for 2-m temperature under different types of land use during June and December shows that the type of land use has a significant impact on the forecast of temperature (Figs. 11a, b), while the types of land use in Northwest China have clear seasonal characteristics. The different characteristics of BE exhibited in different months as analyzed above also indicate that the land-use type in the model does not have seasonal differentiation; meanwhile, the accuracy of the land-use description also has some influence on the temperature forecast.

Figures 11c and 11d show the statistics of TMBE for the forecast of wind speed under different types of land use. It is clear that MBEs in D02 and D03 for different types of land use in June and December are not considerably different. In particular, TMBE of wind speed in D02 is slightly smaller than in D03, and TMBE for barren land is smaller than that for other types of land use. TMBE in December for all types of land use is smaller than that in June.

Figures 11e and 11f show the statistics of TMBE for the forecast of wind direction under different types of land use. In June, except for irrigated cropland and farmland/grassland mosaic, where TMBE of wind direction is positive in D02, TMBE of wind direction under other types of land use is negative, and it is larger for barren land. The largest TMBE of wind direction occurs over barren land, followed by farmland and grassland, and TMBE of wind direction is smaller for other types. TMBE of wind direction forecasted in December is generally higher than that in June. In particular, except for shrub land in D03, where TMBE is the smallest, TMBE is largest under the other types of land use. The relative error of wind direction forecasted in the two domains for shrub land is smaller than for other types, followed by farmland/grassland mosaic and grassland, and TMBE of wind direction is larger for irrigated cropland.

Overall, the forecast of 2-m temperature and wind direction in D03 exhibits a larger error than in D02 for certain types of land use. One possible reason is that wind direction is more subject to the influences of local environment than are temperature and wind speed. As for D03, because the horizontal resolution of this domain is higher, the depiction of land use is more elaborate, and the interpolation of TMBE to stations might generate certain error. In addition, D03 uses the forecast field of D02 as driving data, which may also introduce the forecast error of D02, resulting in larger forecast TMBE in D03 for temperature and wind direction.

In summary, the forecast model for surface variables is sensitive to the types of land use, and accurate description of land use in the model determines the accuracy of forecast of surface variables. However, the current description of land use in China, especially Northwest China, is not sufficiently accurate. For example, the western region of Inner Mongolia is mainly barren with sparse vegetation, but it is described as a farmland/forest mosaic in the model. Therefore, it is necessary to use the newest remote-sensing data and to update the land-use data in the model to improve the forecast results.

Different types of land use clearly differ in the forecast of diurnal variation in near-surface variables. In June (Figs. 12a, c, e), in three types of land use (dryland farms, irrigated cropland, and cropland/grassland), TMBE of 2-m temperature is positive diurnally. The time of maximum MBE is 0800 BT, and there is no difference between daytime and nighttime for shrub land and barren land. TMBE exhibits characteristics of relatively obvious diurnal variation; namely, TMBE is negative in the daytime and positive at night. In grassland, TMBE of 2-m temperature is negative throughout the day, and the absolute value of TMBE is higher than in other situations. In December (Figs. 12b, d, f), TMBE of 2-m temperature has more obvious diurnal variation than in June. Except for dryland farms and grassland, TMBE of 2-m temperature under all other situations exhibits the characteristics of diurnal variation: it is negative in the daytime and positive at night. Under the four types of land use, the time of the maximum absolute value of TMBE of 2-m temperature in the daytime is 1700 BT, and it is the largest for barren land, followed by cropland/grassland and then shrub land. The minimum absolute value of TMBE occurs over irrigated cropland, and the maximum TMBE occurs at night at 0800 BT. The nighttime evolution of TMBE is the same as in the daytime. This indicates that with barren land, the weakening effect of the model forecast on the diurnal variation of temperature is strongest, followed by cropland/grassland, and it is weakest for irrigated cropland. With grassland, however, TMBE is negative throughout the day, and the absolute value of the error is the largest under this type of land use. The maximum absolute error is 6.52°C, which occurs at 1700 BT, and the minimum absolute error is 2.37°C.

Figure 12.  Diurnal variation in TMBE for different variables under various types of land use: (a, b) 2-m temperature; (c, d) 10-m wind speed; and (e, f) 10-m wind direction in (a, c, e) June and (b, d, f) December 2015.

TMBE of wind speed in June under different types of land use is positive throughout the day. With barren land, TMBE of wind speed is at a minimum during 2000–2300 BT and a maximum at 0500 BT. For grassland, TMBE is smallest at 1700 BT and largest at 2300 BT; in other situations, TMBE of wind speed reaches a maximum during 1700–2000 BT and is smallest at 0800 BT. In terms of TMBE magnitude, TMBE is sequentially ranked from large to small with shrub land, cropland/grassland, irrigated cropland, dryland farms, grassland, and barren land. In December, the diurnal variation in TMBE of 2-m temperature is more significant: TMBE is essentially negative in the daytime and positive at night, indicating that the weakening effect of the model on the diurnal variation in temperature during December is less intense than that in June.

As mentioned in the daily evolution of BE for different variables in Section 4.2, weather processes and their strength have significant influences on the forecast of surface variables. Therefore, in this section, cases of strong and weak weather processes in June and December are selected to validate and assess the forecast accuracy of surface variables. The selection criteria for cases with strong and weak weather processes are described by Zhang et al. (2013). A strong weather process is a weather system with a cold front, closed low, trough, low pressure system, or higher wind speed (> 5 m s^–1) at 700 (or 850) hPa, while a weak weather process refers to a high pressure system, ridge, or low wind speed (< 5 m s^–1) at 700 (or 850) hPa. The selected strong and weak weather processes in June and December are listed in Table 5, and all of the following analyses are based on the results in D03.

Figure 13 shows the diurnal variation of variables during strong and weak weather. The 2-m temperature shows obvious diurnal variation in both strong and weak weather in June, but the diurnal variation in strong weather is weaker than that in weak weather. When the weather is strong, daytime 2-m temperatures rise slowly, and the maximum temperature is lower than that of weak weather. At night, due to the presence of clouds and dust aerosols, temperature drops slowly and the lowest temperature is higher than that in weak weather. Meanwhile, in weak weather, 2-m temperature warms faster during the daytime and the maximum temperature is higher. The wind speed increases and is greater than that in strong weather due to the thermal effects of the ground. In addition, the near-surface variables are greatly affected by strong weather, including strong mixing and local influences, which cause the diurnal variation in such weather to be weak. After nightfall, the thermal effect disappears, and the wind speed decreases and is close to that of strong weather. In strong weather in June, the wind direction is mainly southerly and southwesterly, while in weak weather, the winds are mainly southeasterly and southerly.

Figure 13.  Diurnal variations in TMAE for different variables during strong and weak weather: (a, b) temperature; (c, d) wind speed; and (e, f) wind direction in (a, c, e) June and (b, d, f) December 2015.

Diurnal variations of the near-surface variables in December are significantly different from those in June. In strong and weak weather, the diurnal variation of predicted 2-m temperature is weaker than the observed value. In addition, 2-m temperature in strong weather is higher than in weak weather, which is consistent with observations. In both strong and weak weather, although the forecasted 10-m wind speed is generally higher than observation, the diurnal trend of 10-m wind speed is consistent with observation. In addition, the observed 10-m wind direction is mainly west-westerly in strong weather, but the forecasted wind direction is mainly southwesterly; the observed wind direction in weak weather is mainly southwesterly, but the simulation is slightly southeasterly. In winter, 2-m temperature is affected mainly by the advection. The average southerly wind brings warm air, which is also a possible reason for the higher 2-m temperature.

Figure 14 shows a comparison of TMAE for surface variables under different terrain heights during strong and weak weather in June and December. In June, during strong weather, TMAE of 2-m temperature decreases as the terrain height increases. When terrain height is > 3000 m, TMAE of temperature is < 1°C; when terrain height is < 1000 m, it is close to 2°C. During weak weather, TMAE of temperature at different terrain heights is not considerably different, which indicates that the correlation between TMAE of 2-m temperature and the terrain height is low during weak weather. But this is not the case in December. During both strong and weak weather, TMAE of 2-m temperature increases as the terrain height increases; particularly, when terrain height is > 2000 m, TMAE of 2-m temperature can reach 2.4°C.

Figure 14.  TMAE of surface variables at different terrain heights during strong and weak weather: (a, b) 2-m temperature; (c, d) 10-m wind speed; and (e, f) 10-m wind direction in (a, c, e) June and (b, d, f) December 2015.

As for TMAE of 10-m wind speed, during strong weather in June, TMAE is not considerably different at different terrain heights. However, during weak weather, TMAE is clearly different at different terrain heights. When terrain height is < 2000 m, TMAE is > 2.4 m s^–1; when the terrain is > 2000 m, TMAE is < 1.5 m s^–1. The variation of TMAE in December is opposite to that in June. During days without weather processes, TMAE of wind speed is not considerably different, and the average value is < 1.2 m s^–1. However, when there are weather processes, TMAE of wind speed at terrain heights of < 2000 m is larger than that at high terrain heights of > 2000 m, and the TMAE is above 2.0 m s^–1. In addition, when the terrain height is > 2000 m, TMAE of wind speed is essentially consistent under both strong and weak weather conditions. This indicates that TMAE of the wind speed predicted by the model is relatively stable for terrain heights > 2000 m, and strong and weak weather processes do not have a considerable influence on the forecast of wind speed. Figures 14e and 14f compare TMAE of the 10-m wind direction. It is clear that TMAE of 10-m wind direction under strong weather is generally greater than that during weak weather. During strong weather, TMAE of the 10-m wind direction is not considerably different at different terrain heights, and it is larger (almost 50°) when the terrain height is > 3000 m. During weak weather, however, TMAE of wind direction is less than 40° only at heights of 2000–3000 m. TMAE of wind direction is greater than 50° at other heights, and it is the largest (> 60°) at 1000–2000 m. The opposite is true in December. During strong weather, TMAE of wind direction is smaller than that during weak weather. It is smaller (< 30°) over 2000–3000 m; at different heights, it is always greater than 50°. In weak weather, TMAE of wind direction is smaller than 35° at all heights.

Overall, there are clear differences in the characteristics of TMAEs for surface variables under strong and weak weather for various terrain heights in June and December. Generally speaking, during weak weather in June, for TMAE of 10-m wind speed and wind direction, it is larger when the terrain height is relatively low, while the opposite is true in December. During strong weather in June, when the terrain is relatively low, TMAE of the temperature is higher; in December, however, when the terrain is higher, TMAE of the temperature is higher. In addition, during weak weather in June, the terrain height does not have a significant impact on the temperature forecast, while the TMAE of temperature forecast in December increases as the terrain height increases.

According to the values of TMAE of predicted 2-m temperature during different weather processes under different types of land use, TMAE of 2-m temperature in June during strong weather is higher in dryland farms, irrigated cropland, and farmland/grassland mosaic, and the value is greater than 2°C (Fig. 15). In particular, TMAE of the dryland farms is the largest, reaching 2.5°C, while TMAE under other types of land use is below 1.5°C. During weak weather, TMAE of 2-m temperature does not change considerably under different types of land use, and it is essentially below 1.6°C. However, in Dec-ember, TMAE of 2-m temperature under different types of land use is not considerably different during strong weather and is essentially below 2.0°C. TMAE is larger (almost 2.2°C) only when the land is barren. The variation of TMAE during strong weather is clearly different from that during weak weather. That is, except for dryland farms and irrigated cropland, for which TMAE of the 2-m temperature is small (< 1.2°C), TMAE of 2-m temperature for other types of land use is larger, essentially greater than 2.4°C.

Figure 15.  TMAE of surface variables under different types of land use during strong and weak weather: (a, b) 2-m temperature; (c, d) 10-m wind speed; and (e, f) 10-m wind direction in (a, c, e) June and (b, d, f) December 2015.

Regarding TMAE of 10-m wind speed in different weather processes, TMAE of 10-m wind speed with different types of land use in June during weak weather is larger than that during strong weather. In particular, the value of TMAE is higher for dryland farms and shrub land and is greater than 2.4 m s^–1. During strong weather, TMAE for different types of land use is not significantly different. It is greater than 2.0 m s^–1 only for grassland and shrub land, and smaller than 2.0 m s^–1 for other types of land use. The case is opposite in December. During strong weather, TMAE of 10-m wind speed for different types of land use is greater than that for weak weather. In particular, in farmland/grassland mosaic and shrub land, TMAE of 10-m wind speed is greater than 2.3 m s^–1, and TMAE for other types of land use is less than 2.0 m s^–1; in weak weather, TMAE is smaller for different types of land use, and it is less than 1.6 m s^–1.

For TMAE of 10-m wind direction, values during strong weather are greater than those during weak weather in June in farmland/grassland mosaic. The TMAE during weak weather is greater than that during strong weather for other types of land use. In addition, for irrigated cropland, the error of strong and weak weather is smaller than for other types. December is different from June. Under different types of land use, TMAE of 10-m wind direction for strong weather is higher than that for weak weather. For farmland/grassland mosaic, TMAE reaches the largest of 120°, and for irrigated cropland and barren land it is less than 70°. During weak weather, TMAE is the smallest for irrigated cropland and is close to 20°; it is the largest in grassland and shrub land and is more than 70°.

In summary, there are clear differences in the variation of the TMAE of surface variables during different weather processes for different land-use types in June and December. Generally speaking, the variations in 10-m wind speed and wind direction are opposite in June and December. That is, TMAE of 10-m wind speed and wind direction during strong weather is smaller than that during weak weather in June, while the opposite is true in December. The variation of TMAE for 2-m temperature is different from that for wind speed and direction. During strong weather in June, the values of TMAE of 2-m temperature in dryland farms, irrigated cropland, and farmland/grassland mosaic are higher, while they are relatively low in December. In addition, during weak weather in December, in farmland/grassland mosaic, grassland, shrub land, and barren land, TMAE of 2-m temperature is larger. A possible reason for the different characteristics of TMAE in December and June is the seasonal change in land-use types in Northwest China; namely, the irrigated cropland, cropland, and grassland in June become barren and sparse vegetation in December, and there are also changes in the corresponding land surface processes. The model, however, does not reflect these seasonal changes in land use; therefore, it causes clear differences in the TMAE of surface variables under various types of land use in December and June for different weather processes. In a word, seasonal changes in land use influence the forecast of surface variables in different seasons, which is an important aspect for future improvement of the model.

In this study, we validated and evaluated the near-surface variables (including 2-m temperature and 10-m wind speed and wind direction) predicted by the WRF model for different seasons, different terrain heights, different land-use types, and various weather situations in Northwest China, in regions with complex terrain. Spatial distribution of the monthly average error of June and December is analyzed first. It is found that the spatial distribution of the 2-m temperature forecast error from the WRF model is correlated with the terrain distribution in June. That is, a positive deviation of temperature is apparent in deserts and basins where evaporation is high, while negative deviation exists in high terrain. However, this feature is not obvious in December. In addition, the model can capture the diurnal variation characteristics of 2-m temperature, but with lower predictions for high temperature and higher predictions for low temperature; that is, this simulation weakened the diurnal variation of temperature, which is consistent with the results of Zhang et al. (2013).

Variations in the errors of the near-surface variables at different terrain heights and with various types of land use reveal that forecast errors are dependent on terrain height and land-use types. Meanwhile, an investigating of the significant differences in the errors of near-surface variables during strong and weak weather in June and December shows that the forecast error of near-surface variables is considerably related to terrain, land use and its seasonal changes. This is due to the difference in the terrain height and land use between the model and the actual situation. It is also indicated that the inaccurate description of underlying surface states in the model has restricted the model’s ability in forecasting near-surface variables. Therefore, the reason for the generation of errors in near-surface variables merits further analysis. Meanwhile, precise description of terrain height, the updating and replacement of land-use descriptions, and consideration of seasonal land-use changes are important aspects for future model improvements.

Acknowledgments We appreciate the constructive comments and suggestions from the two anonymous reviewers.