Visualization Control System (VCS) Basic Tutorial ¶
VCS allows scientists to produce highly customized plots. This tutorial provides an overview of VCS while other tutorials on the CDAT Tutorials page provide more specifics on additional VCS Principles, the VCS Template which controls many aspects of a plot, Text Objects in VCS and another example of using VCS to plot data from start to finish in the VCS Example tutorial. We recommend you also look at the CDMS 101 notebook.
The most direct way to work with this Jupyter Notebook is to download the notebook by right-clicking on the link below and chosing "Download Linked File As..." or "Save Link as...", activating a CDAT + Jupyter compatible environment, and running the notebook on its own or within a JupyterLab interface. For more details, see the Getting Started section below.
If you are unfamiliar with Jupyter Notebooks, they are files with an .ipynb extension that are made up of cells that can include executable code or regular text to explain what the code is doing. From Jupyter.org "The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text." Users can step through each cell in the notebook by putting their cursor in the cell they wish to run and either clicking on the "Run" button at the top of the page or pressing enter on the keyboard while holding down shift at the same time (shift-enter).
To download this Jupyter Notebook, right click on the link and choose "Download Linked File As..." or "Save Link as...". Remember where you save the downloaded file which should have an .ipynb extension. (You'll need to launch the Jupyter notebook or JupyterLab from the location where you store the notebook file.)
Getting Started¶
This notebook uses Python 3.
If you only have Python 2 installed, the code should still work in Python 2, but if it doesn't, isn't this a good time to join the Python 3 crowd? The instructions below make it relatively painless to install a Python 3 environment.
For the code in this notebook to work, you need to install a CDAT compatible environment. See the next cell in the notebook for details on how to create an appropriate environment and activate it. If you see three grey dots, that cell has been hidden. Just click on the dots to see the contents of the cell. (To hide the cell in JupyterLab, click on the vertical blue bar to the left of the cell.)
Conda Installation¶
We recommend using conda to install and manage environments. Conda itself can be installed either via Anaconda (the everything-but-the-kitchen-sink version) or Miniconda (the minimalist version).
I prefer Miniconda since it takes up less room on my computer and I don't mind installing the various packages I need when I need them. Miniconda (or Anaconda) will install Python on your computer. If you already have Python installed on your computer, see this helpful page. The choice of whether to install the Python 2.x or Python 3.x version of Miniconda will affect only your root environment. You can create both Python 2.x and Python 3.x environments with either version of Miniconda installed (and many environments can exist on your computer at the same time).
When installing Miniconda or Anaconda, let the installer add the conda installation of Python to your PATH environment variable. On a Mac or Linux machine, if asked "Do you wish the installer to initialize Anaconda3 by running conda init?" or something similar, we recommend saying "yes". For more details see this page.
Create a CDAT Compatible Environment¶
Once you have a version of Miniconda (or Anaconda) installed, create a CDAT or Jupyter-VCDAT environment.
For CDAT¶
Type the following code at a command line prompt. Note, since all output of code in this notebook will be displayed within the notebook, we do not need a separate display window so we can use the mesalib (or headless display) version of the latest version of the cdatX.x environment, which is currently cdat8.1.
conda create -n cdat81-mesa -c cdat/label/v81 -c conda-forge python=3.6 cdat mesalibOnce the cdat81-mesa environment is installed, activate it by typing:
conda activate cdat81-mesaInstall JupyterLab within your cdat81-mesa environment so you can run Jupyter notebooks:
conda install -c conda-forge jupyterlab
For additional details on creating a CDAT environment, see the following CDAT installation page.
For VCDAT and the jupyter-vcdat environment¶
Follow these instructions for installing VCDAT 2.0 on your personal computer.
Once the jupyter-vcdat environment (and VCDAT) have been installed, activate the environment with:
conda activate jupyter-vcdat
If "conda activate" doesn't work¶
Try:
source activate name_of_environmentStart JupyterLab¶
Once you have a cdatXx-mesa or jupyter-vcdat environment activated, navigate to the parent folder that contains this notebook, then type:
jupyter-labNote: you must launch JupyterLab from the highest level folder you want to be able to access. Jupyter can see folders below the directory from which it was launched, but it cannot see directories above its launch directory.
If you do not want to use the enhanced JupyterLab interface, you can run:
jupyter-notebookto load only the notebook without the JupyterLab interface.
Once JupyterLab (or Jupyter Notebook) has started, you may be asked to pick a Kernel. Choose the generic Python 3 environment or cdatXx-mesa or jupyter-vcdat. If you chose the generic Python 3 environment that means Jupyter will use whichever Python environment started the JupyterLab (or Jupyter Notebook) session. Hence it is best to start the JupyterLab session from an environment that contains CDAT.
As a side note, VCDAT and VCS are two different CDAT modules though they have similar acronyms. VCDAT 2.0 helps you visualize, manipulate data, and test out code within the JupyterLab interface. VCS stands for Visualization Control System which allows you to create customized plots and animations.
Download Sample Data ¶
First let's download some sample data, specifically the three NetCDF files: ta_ncep_87-6-88-4.nc, clt.nc, and sampleCurveGrid4.nc, which will be stored in the same directory as this notebook.
from __future__ import print_function
import requests
import os
for filename in ['ta_ncep_87-6-88-4.nc', 'clt.nc', 'sampleCurveGrid4.nc']:
if not os.path.exists(filename):
r = requests.get("https://cdat.llnl.gov/cdat/sample_data/{}".format(filename), stream=True)
with open(filename,"wb") as f:
for chunk in r.iter_content(chunk_size=1024):
if chunk:
f.write(chunk) # write this chunk to a local version of the file
Opening Data Files ¶
First we will open one of our demo files, which is done via the cdms2 module.
In the output below the code cell below, if you see a message that says "Allow anonymous logging usage to help improve CDAT(you can also set the environment variable CDAT_ANONYMOUS_LOG to yes or no)? [yes]/no:" either press enter (for yes) or type yes or no. If you do not respond with a return or yes or no, you will not be able to proceed.
import cdms2
f = cdms2.open("ta_ncep_87-6-88-4.nc")
f.variables.keys()
We can obtain information on a specific variable using the info method. In the next line of code we look at the ta variable which is the Air Temperature in degrees Kelvin (K). The file contains:
- 11 months of Air Temperature data (the T axis),
- 17 different atmospheric pressure levels (the Z axis),
- 73 latitude bands (spanning from 90 to -90) (Y axis) and
- 144 longitude bands (spanning from 0 to 360) (X axis).
The latitude axis is Y and the longitude axis is X. (If this seems backwards, think of the values of latitude increasing or decreasing along the Y axis though the actual lines of latitude - or parallels - follow the X axis).
As you'll see in the output, the shape of the data is (11, 17, 73, 144). Each axis is also known as a dimension.
f["ta"].info()
In general, with the latitude dimension having a length of 73 (meaning there are 73 temperature values from -90 to 90 degrees) and the longitude dimension having a length of 144 (or 144 temperature values from 0 to 360 degrees), a single temperature value spans a 2.5 x 2.5 degree portion of the globe. The exception is in the latitude dimension which would normally have a length of 72 if all sections of the grid were 2.5 degrees. Instead, it has a length of 73 because the first and last grid elements (at -90 and 90 degrees) are each 1.25 degrees tall not 2.5. You can see this by displaying the latitude slices in the data using the getBounds() method.
f["ta"].getLatitude().getBounds()
First Plot ¶
Now, let's visualize the ta variable. For this we will need the vcs module.
We will also need a vcs canvas upon which to plot the data.
import vcs
canvas = vcs.init()
canvas.plot(f["ta"]) # This plots the first time interval (1987/6/1) and the first atmospheric level (1000).
Subsetting Data¶
In the above plot we used all the variable dimensions (time, level, latitude and longitude) and the resulting plot is a boxfill plot. (If the data has at least two dimensions for the variable being plotted, the default plot type is boxfill.)
You can load and use a subset of the dimensions, however. Notice how we can use either the actual dimension value (e.g. -90 for longitude) or indices (via slice). For this dataset, the longitude values range from 0 to 360, so specifying a -90 value for longintude means 360-90 = 270.
data = f["ta"](longitude=-90, latitude=40, level=500, time=slice(0,1))
print(data)
In the previous example, we fixed all dimensions and the command returned a single value. Let's subset longitude to a range rather than a single value and plot it.
The "squeeze" parameter in the first line below prevents dimensions of length 1 (e.g. latitude, level, and time) from being plotted.
data = f["ta"](longitude=(0,180), latitude=40, level=500, time=slice(0,1), squeeze=1)
canvas.clear()
canvas.plot(data)
Using Isolines¶
If we want an isoline rather than the default boxfill, we will need to create the associated isoline graphic rendering object using vcs' createisoline method.
isoline = vcs.createisoline()
data = f("ta")
canvas.clear()
canvas.plot(data, isoline)
canvas.clear()
data = f["ta"](longitude=-90, latitude=(-90,90), level=(1000,100), time=slice(0,1), squeeze=1)
canvas.plot(data, isoline)
We can also colorize the isolines, if we want, and turn on labels.
levels = vcs.mkscale(*vcs.minmax(data))
colors = vcs.getcolors(levels)
isoline.levels = levels
isoline.linecolors = colors
isoline.label = True
isoline.textcolors= colors
canvas.clear()
canvas.plot(data, isoline)
Easy Plot Tweaks¶
We can also control the dimensions' "range" on the plot via the isoline graphic rendering object,
data = f["ta"](longitude=-90, time=slice(0,1), squeeze=1)
# The next two lines put the North Pole on the left of the plot
isoline.datawc_x1 = 90
isoline.datawc_x2 = -90
isoline.datawc_y1 = 3 # log10(1000)
isoline.datawc_y2 = 2.3 # log10(200)
Or tweak the axes' scale (turn the y axis of the plot (not the data) - the atmospheric pressure levels - into a base 10 logarithmic scale),
isoline.yaxisconvert = "log10"
Or change the default labels.
isoline.yticlabels1 = {1000:"1000", 800:"800", 500: "500", 20:"20"}
canvas.clear()
canvas.plot(data, isoline)
Overlays¶
Now let's overlay another variable from a different NetCDF file: the u component of the wind from the clt.nc file. Since u has different values than the previous ta variable, we will create a separate isoline graphic rendering object, iso2.
f2 = cdms2.open("clt.nc")
u = f2("u", longitude=-90., time=slice(0,1), squeeze=1) # First time slice
iso2 = vcs.createisoline(source=isoline) # Essentially makes a copy
levels = vcs.mkscale(*vcs.minmax(u))
iso2.levels = levels
colors = ["blue",] # Blue lines
iso2.linecolors = colors
iso2.textcolors = colors
canvas.clear()
canvas.plot(data, isoline)
canvas.plot(u, iso2)
Manipulating Variables¶
Manipulating variables is easy. Let's plot the temperature in Fahrenheit rather than Kelvin.
As the .info() command shows, even though we've changed the values of the ta variable to be in Fahrenheit, the title still says the Air Temperature is in Kelvin, the units are not specified, and everything else remains the same as before the conversion into Fahrenheit.
ta_f = (f["ta"]-273.15)*9/5+32
ta_f.info()
The next two lines specify that the units are Fahrenheit (for plotting purposes) and adjusts the title to match the data. $^o$F is the sequence of characters that is needed to properly plot the degree symbol and "F" on the image ($^o$F). When we type ta_f.info(), we can see that the units and title have been updated.
ta_f.units = r"$^o$F"
ta_f.title = r"Air Temperature ($^o$F)"
ta_f.info()
canvas.clear()
canvas.plot(ta_f)
We can also combine variables together or perform other calculations. The v variable comes from the already open clt.nc NetCDF file and is the v component of the wind.
u = f2("u")
v = f2("v")
speed = cdms2.MV2.sqrt(u**2 + v**2)
speed.id = "Wind Speed"
speed.units = "m/s"
canvas.clear()
canvas.plot(speed)
import genutil, cdutil
data = f("ta", time=slice(0,5))
ta_avg = genutil.averager(data, axis='t')
canvas.clear()
canvas.plot(ta_avg)
Remove Zonal Mean¶
We can also remove the zonal mean by averaging over longitude. For each latitude slice in the data, the averager function below averages the tempeartures over all the longitudes and stores a single averaged value for that latitude. Hence, after averaging ta_zm no longer has longitude values.
ta = f("ta")
ta_zm = genutil.averager(f("ta"), axis="x")
ta_zm.shape
To remove the zonal mean we need to subtract ta_zm from ta, but to do that the two need to be the same size and shape (11, 17, 73, 144). Hence we need to grow ta_zm in the longitudinal direction. This will take the average temperature for each latitude band and spread it across all 144 longitude values in the file to match the dimensions of ta.
ta, ta_zm = genutil.grower(ta, ta_zm)
ta_zm.shape
canvas.clear()
canvas.plot(ta-ta_zm) # the zonal mean has been removed.
Time Difference¶
Next, we can do a simple time difference (last - first time point).
diff = ta[-1] - ta[0]
diff.shape
Time Offset¶
We can offset all points by one day.
offset = ta[1:] - ta[:-1]
offset.shape
Note that the other axes are preserved.
offset.getAxisList()
Other graphic methods¶
Now we will move on to the topic of controlling the graphics output. So far, we have usually allowed VCS to chose the defaults.
Isofill¶
We have already seen the default boxfill and how to use isoline. Another available plot type combines these two and is called isolfill. In the example below, the isofill object is called gm (for graphic method) and is generated using the createisofill method. The plot covers North America, Greenland and the North Atlantic at the 500 millibar level and the first time value of 1987/6/1 or June 1, 1987.
ta = f("ta", longitude=(-180, 0), latitude=(0,90), level=500, squeeze=1)
gm = vcs.createisofill()
canvas.clear()
canvas.plot(ta, gm)
Vectors ¶
The vector plot type requires two variables. Here they are the u and v components of the wind, subsetted to show North America. The vector graphic rendering object (gm) is generated via the createvector method.
u = f2("u", longitude=(-180, 0), latitude=(0,90), squeeze=1)
v = f2("v", longitude=(-180, 0), latitude=(0,90), squeeze=1)
gm = vcs.createvector()
canvas.clear()
canvas.plot(u, v, gm)
Streamlines ¶
The streamlines plot type also requires two variables. Here we are using the u and v components of the wind, as we defined and subsetted them in the previous step.
gm = vcs.createstreamline()
canvas.clear()
canvas.plot(u,v, gm)
Meshfill - Using Alternative Grids ¶
If your NetCDF file is CF compliant, VCS can plot generic grids right out of the box. Here we are using a sample curvilinear grid. The sample variable in the NetCDF file ("sample") is assigned to the variable name curv, and m is the meshfill graphic rendering object created via the createmeshfill method.
f3 = cdms2.open("sampleCurveGrid4.nc")
curv = f3("sample")
m = vcs.createmeshfill()
The first plot we will look at is the raw data that makes up the curviliniear grid. The grid itself is 32 rows by 48 columns.
The sample data that fills this grid were created to have some data to plot. The data values were generated by starting with a low value of 0.0 in the 0,0 location and increasing to a maximum value of 1535.0 in the last row and column. If we plot this data using 32 rows and 48 columns we get the following image which has the highest value of 1535.0 in the upper right-hand corner.
canvas.clear()
canvas.plot(curv.filled())
The purpose of this curvilinear grid is for modeling the oceans, so this grid is wrapped around the Earth in such a way that the North Pole is over land rather than over the Arctic Ocean. Hence in the plot below the "North Pole" of the grid - the white area where there is no data - is over land, i.e. northern Canada.
canvas.clear()
canvas.plot(curv, m)
Projections ¶
You can also adjust the projection. First create a projection graphic rendering object (proj) by using the createprojection method. Specify the projection type and the standard parallels, then assign that projection to the .projection attribute of your object. Here we've created a boxfill object called gm and assigned it the Lambert projection with the first standard parallel at 0 and the second standard parallel at 89 degrees. We are using the ta object from the Isofill example.
proj = vcs.createprojection()
proj.type = "lambert"
proj.standardparallel1 = 0
proj.standardparallel2 = 89
gm = vcs.createboxfill()
gm.projection = proj
canvas.clear()
canvas.plot(ta, gm)
The CDAT software was developed by LLNL. This tutorial was written by Charles Doutriaux and Holly Davis. This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344.
If you have questions about this notebook, please email our CDAT Support address, cdat-support@llnl.gov.