There are many different plotting packages available for use with Python. This tutorial will use Matplotlib
as its plotting package - so the first thing you need to do is install Matplotlib. If you use Debian or Ubuntu, just "sudo apt-get install python-matplotlib".
There are two approaches to serving up plots and graphs to a user's browser:
- Option 1 - write images to files. If we do this, our software will make a plot and write binary image data to a file. The user's browser then requests this image from the web server, which reads the file and sends binary data to the browser. While this seems simple enough from a programmatic point of view, over time the server disk drive fills up unless you clean up old files - and you must provide unique file names for each session, otherwise users will download each other's image files!
- Option 2 - send binary image data directly from Pylons. In Option 1 we wrote binary data to files; here we send it to the browser directly. No intermediate files to manage and clean up, and it is faster than writing them and then reading them again. Your code never touches the server hard drive.
In this tutorial, we will use Option 2 - send binary image data directly from Pylons. There is a small up-front investment in programming time (which the tutorial does for you), but the payoff is real and is manifested in simplified server management and web site maintenance. It is also pretty cool - you can do this with any image data! To grab the binary image data from Matplotlib, we will use the Python Image Library (PIL) - so download and install PIL. If you use Debian or Ubuntu, just "sudo apt-get install python-imaging".
Now let's make a test to find out if the plotting package installation works with Pylons on your system.
Here we'll make a very simple test that only returns an image, plotting data that we will generate on the fly. Add the following near the top of fitter.py, immediately before the class declaration:
1
2
3
4
5
6
7
8 | import matplotlib
matplotlib.use('Agg')
import pylab, matplotlib.axes3d
import PIL, PIL.Image, StringIO, threading
imageThreadLock = threading.Lock() # make sure methods for graphics do not overwrite each other
timesRequested = 0 # this is only to give a different graph with each refresh
|
Now add a new method to fitter.py:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35 | # this comment forces the wiki to indent correctly, and is not needed
def TestMatplotlibImage(self):
global timesRequested # this is only to give a different graph with each refresh
global imageThreadLock # prevent threads from writing over each other's graphics
# set the response type to PNG, since we at least hope to return a PNG image here
response.headers['Content-type'] = 'image/png'
# make a buffer to hold our data
buffer = StringIO.StringIO()
# lock graphics
imageThreadLock.acquire()
# we don't want different threads to write on each other's canvases, so make sure we have a new one
pylab.close()
canvas = pylab.get_current_fig_manager().canvas
# quick simple plot
pylab.plot([1+timesRequested, 2+timesRequested, 3+timesRequested], [40+timesRequested, 50+timesRequested, 60+timesRequested])
timesRequested += 1 # this is only to give a different graph with each refresh
canvas.draw()
imageSize = canvas.get_width_height()
imageRgb = canvas.tostring_rgb()
pilImage = PIL.Image.fromstring("RGB", imageSize, imageRgb)
pilImage.save(buffer, "PNG") # <-- we will be sending the browser a "PNG file"
# unlock graphics
imageThreadLock.release()
return buffer.getvalue()
|
Note that we set the response type to image/png, so that the browser knows what type of binary data we're sending back. Normally this would be binary data read from an image file on the hard disk drive, but here it is the binary data in our PNG buffer. We also acquire and release threadlocks around our graphics code, so the graphs do not write on top of each other.
At this point a call to http://127.0.0.1:5000/fitter/TestMatplotlibImage should return a simple straight-line plot. Reload a few times to ensure the data shown on the X and Y axes is changing with each refresh.
Now we can add some graphs to our curve and surface fitting web site.
Note that since we are making dynamic plots and graphs, the requests to plot data will arrive at the web server after requests to curve fit have been completed. The sequence goes something like this:
- Our user inputs their data, selects an equation, and submits the form to the web server.
- The web server fits the data and returns HTML to the user's brower - including links to graphs.
- The user's browser reads the HTML we just sent, and makes separate requests for the graphs we linked to.
- The web server generates the graphs, and send them to the user's browser as binary data.
- The user's browser renders all this for the user to view.
Steps 2 and 4 above are independent, so on the server side we must somehow save information between these steps. To do so we will use sessions, which are conveniently provided by Pylons.
Add the following code to the end of the FitAndReturnResults() method in fitter.py so that it looks like this:
1
2
3
4
5
6
7 | # this comment forces the wiki to indent correctly, and is not needed
# save to session for graphing
session['equation'] = c.equation
session.save()
return render('/results.mako')
|
and now add the following code to the end of the FitAndReturnResults3D() method in fitter.py so that it looks like this:
1
2
3
4
5
6
7 | # this comment forces the wiki to indent correctly, and is not needed
# save to session for graphing
session['equation'] = c.equation
session.save()
return render('/results3D.mako')
|
The first graph we'll add is a scatter plot of the fitting error to help us determine the quality of our curve or surface fits. Add the following code to fitter.py:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43 | # this comment forces the wiki to indent correctly, and is not needed
def AbsoluteErrorVsDependentData(self):
global imageThreadLock # prevent threads from writing over each other's graphics
# set the response type to PNG, since we at least hope to return a PNG image here
response.headers['Content-type'] = 'image/png'
# make a buffer to hold our data
buffer = StringIO.StringIO()
# lock graphics
imageThreadLock.acquire()
# we don't want different threads to write on each other's canvases
pylab.close()
# get the equation from the session
equation = session['equation']
# 500x400 pixels, white background
figure = pylab.figure(figsize=(5,4), dpi=100, frameon=False)
# http://matplotlib.sourceforge.net/matplotlib.pylab.html#-scatter
pylab.scatter(equation.DependentDataArray, equation.AbsoluteErrorArray)
# label the axes and give a title
pylab.xlabel('Dependent Data')
pylab.ylabel('Absolute Error')
pylab.title('Absolute Error vs. Dependent Data')
canvas = pylab.get_current_fig_manager().canvas
canvas.draw()
imageSize = canvas.get_width_height()
imageRgb = canvas.tostring_rgb()
pilImage = PIL.Image.fromstring("RGB", imageSize, imageRgb)
pilImage.save(buffer, "PNG") # <-- we will be sending the browser a "PNG file"
# unlock graphics
imageThreadLock.release()
return buffer.getvalue()
|
To show the graph to our users, add this to the bottom of both results.mako and results3D.mako, just before the final </PRE> tags:
1 | <img src='/fitter/AbsoluteErrorVsDependentData'>
|
and again try http://127.0.0.1:5000/ and http://127.0.0.1:5000/fitter/index3D - you should now have scatter plots of error vs. dependent data.
Now let's add a histogram of absolute error. Add the following code to fitter.py:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43 | # this comment forces the wiki to indent correctly, and is not needed
def AbsoluteErrorHistogram(self):
global imageThreadLock # prevent threads from writing over each other's graphics
# set the response type to PNG, since we at least hope to return a PNG image here
response.headers['Content-type'] = 'image/png'
# make a buffer to hold our data
buffer = StringIO.StringIO()
# lock graphics
imageThreadLock.acquire()
# we don't want different threads to write on each other's canvases
pylab.close()
# get the equation from the session
equation = session['equation']
# 500x400 pixels, white background
figure = pylab.figure(figsize=(5,4), dpi=100, frameon=False)
# http://matplotlib.sourceforge.net/matplotlib.pylab.html#-hist
pylab.hist(equation.AbsoluteErrorArray, 10)
# label the axes and give a title
pylab.xlabel('Absolute Error')
pylab.ylabel('Frequency')
pylab.title('Histogram of Absolute Error')
canvas = pylab.get_current_fig_manager().canvas
canvas.draw()
imageSize = canvas.get_width_height()
imageRgb = canvas.tostring_rgb()
pilImage = PIL.Image.fromstring("RGB", imageSize, imageRgb)
pilImage.save(buffer, "PNG") # <-- we will be sending the browser a "PNG file"
# unlock graphics
imageThreadLock.release()
return buffer.getvalue()
|
To show the graph to our users, as before we add this to the bottom of both results.mako and results3D.mako, just before the final </PRE> tags:
1 | <img src='/fitter/AbsoluteErrorHistogram'>
|
and again try http://127.0.0.1:5000/ and http://127.0.0.1:5000/fitter/index3D - you should now have histograms of absolute error.
Now for our final graphic image, a 3D scatterplot of user data. Add the following code to fitter.py:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44 | # this comment forces the wiki to indent correctly, and is not needed
def ScatterPlot3D(self):
global imageThreadLock # prevent threads from writing over each other's graphics
# set the response type to PNG, since we at least hope to return a PNG image here
response.headers['Content-type'] = 'image/png'
# make a buffer to hold our data
buffer = StringIO.StringIO()
# lock graphics
imageThreadLock.acquire()
# we don't want different threads to write on each other's canvases
pylab.close()
# get the equation from the session
equation = session['equation']
# 500x400 pixels, white background
figure = pylab.figure(figsize=(5,4), dpi=100, frameon=False)
# http://www.scipy.org/Cookbook/Matplotlib/mplot3D
ax = matplotlib.axes3d.Axes3D(figure)
ax.scatter3D(equation.IndependentDataArray[0], equation.IndependentDataArray[1], equation.DependentDataArray)
# label the axes and give a title
ax.set_xlabel('X data')
ax.set_ylabel('Y data')
ax.set_zlabel('Z data')
canvas = pylab.get_current_fig_manager().canvas
canvas.draw()
imageSize = canvas.get_width_height()
imageRgb = canvas.tostring_rgb()
pilImage = PIL.Image.fromstring("RGB", imageSize, imageRgb)
pilImage.save(buffer, "PNG") # <-- we will be sending the browser a "PNG file"
# unlock graphics
imageThreadLock.release()
return buffer.getvalue()
|
To show the 3D plot to our users, we add this to the bottom of results3D.mako, just before the final </PRE> tags:
1 | <img src='/fitter/ScatterPlot3D'>
|
Now when you try http://127.0.0.1:5000/fitter/index3D, the results should show a 3D scatterplot of the user data.
Next: --> Conclusion
