I have mentioned before how one can access data files on your hard drive, but Python also allows you to access remote data, for example on the internet using the HTTP protocol. This is the protocol with which web servers and web browsers exchange information. The easiest way to do this is to use the requests module. To start off, you just can get the URL:
import requests
response = requests.get('http://xkcd.com/353/')
response
holds the response now. You can access the content as text via the text-property:
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
print(response.text[:300]) # only print the first 300 characters
You can either just use this information directly, or in some cases you might want to write it to a file. Let's download one of the full resolution files for the Ice coverage data from Problem Set 2:
r2 = requests.get('http://mpia.de/~robitaille/share/ice_data/20060313.npy')
r2.headers
requests.get?
r2.text[:200]
However, this doesn't seem to be actual text. Instead, its a binary format. The binary data of the response can be accessed via
r2.content[:200]
Note the little b
at the beginning indicating a binary byte-string.
Now we can open a new (binary) file and download the data to the file.
with open('20060313.npy', 'wb') as f:
f.write(r2.content)
Let's now load and plot the data:
import numpy as np
data = np.load('20060313.npy')
%matplotlib inline
import matplotlib.pyplot as plt
plt.figure(figsize=(12,12))
plt.imshow(data, origin='lower')
Imagine that you want to access some data online. A number of websites now offer an "Application programming interface" (or API) which is basically a way of exchanging information in a machine-readable way.
Above, we used the HTTP protocol which is used for accessing web resources. On top of the protocol sits the API that defines how information is to be exchanged for a particular service. E.g., Google describes in detail its API for making search requests to the Google web servers. You may can imagine that the definition of an API can be an extensive document (and it typically is). Moreover, the data formats have to be specified, e.g., if images are exchanged.
An example of a web service with a particularly simple API is http://api.open-notify.org which we will use in a moment. We will make a request to a service at this site that provides the current location of the International Space Station (ISS).
import requests
response = requests.get("http://api.open-notify.org/iss-now.json")
print(response.status_code) # print the status code of the response.
As we have seen before data of the request are stored in the .text attribute:
print(response.text)
print(type(response.text))
This looks pretty simple, like a Python dictionary. However, it is a string (in fact in JSON "JavaScript Object Notation") that we can easily convert to a dictionary:
response = requests.get("http://api.open-notify.org/iss-now.json")
d = eval(response.text) # eval() executes a string as if it is a
# Python command
for i in d:
print(i,' ',d[i])
The service http://api.open-notify.org/astros.json provides information of people who are currently in space. Access the service with Python and print a list of space-people and their space-craft(s).
# your solution here
Applications on the Linux desktop communicate with each other using something called the dbus. Python provides a module that allows us to use the dbus protocol to communicate with applications, here with the planetarium program kstars. For this kstars must be installed and running on your desktop.
import dbus
bus = dbus.SessionBus() # establish connection to dbus demon
remote_object = bus.get_object("org.kde.kstars", "/KStars")
# org.kde.kstars: application name
# /KStars: object path, since application might export many objects
# remote_object.Introspect() # get info on available methods
# generates a namespace
iface = dbus.Interface(remote_object, 'org.kde.kstars')
# direct the application to center the star 'Vega'
iface.lookTowards('Vega')
Here a more complex request accessing the data base of Messier objects stored in kstars. Since the response is in XML format there is a helper function extrating one particular XML-tag.
# helper function
def getxmltag(xstring, name):
d = xstring.rsplit('<'+name+'>')
d = d[1].rsplit('</'+name+'>')
return d[0]
# request data for all 110 Messier objects
for i in range(110):
messier = 'M ' + str(i+1)
a = iface.getObjectDataXML(messier)
print(messier, ' : ', getxmltag(a, 'Constellation'), ' : ',
getxmltag(a,'Type'), ' : ', getxmltag(a,'Magnitude'))