1

The context of this question is that I am trying to write a program for assist in analysis of data. It should be written in python3, however the kind of data it is for is usually stored in a format that python cannot read. There is a package to the read these data but it is only compatible with python2. In order to the read the data I therefore wanted to write a python2 scripts that reads the file and converts it into a numpy array. This I want to read in my python3 program. (The package in question is axographio1).

In generality what I want is this:

Given a (python2) script like

#reading.py
import numpy
x = numpy.random.random(size=10000)

run a (python3) that can somehow get x

#analyze.py
import matplotlib.pyplot as plt
#fancyfunction that gets x from reading.py
plt.plot(x)
plt.show()

It is important here that reading.py be executed by the python2 interpreter since it will not work with python3.

Nikolai
  • 21
  • 2
  • 1
    why not doing everything in python 2.7 ? An other option is having a look here : https://stackoverflow.com/questions/3781851/run-a-python-script-from-another-python-script-passing-in-args. And as third option is to do it in two time : 1- in python2 read your data and write a new file (.txt, .dat, etc...), 2-read this file in python3 script, 3-write a small shell script that will call both to make it more easy for the user. – Dadep Jun 20 '17 at 15:54

2 Answers2

6

Have you tried to pickle the data

In python 2

import pickle
pickle.dumps(x)

In python 3

import pickle
pickle.load(x)

If I remember correctly, a better approach is to save your numpy array in a json file (maybe using panda over numpy) in python 2, and do the reverse in python 3

Something like that :

df = pandas.Data[...]

See http://www.thegeeklegacy.com/t/how-to-import-without-using-import-in-python/35/#post-103 for the details

Axel Der
  • 421
  • 3
  • 10
  • Thanks for your answer. I thought about storing the data in a file but wanted to avoid if possible. – Nikolai Jun 24 '17 at 14:51
  • Anyway, I will accept your answer if you update it a little so that it could be run. (pickle.dumps take more than one argument afaik) – Nikolai Jun 24 '17 at 14:52
0

Below is the solution I used. From the subprocess module I used the function Popen with argument shell=True to call my python2 scripts from the python3 and collected the stdout. (This means everything has be printed to the console.)

This is python2 code I want to call in python3:

#readying.py
import axographio #this is the python2 package
import os #specific to the problem of reading data
import sys #needed to use arguments from the command line
if __name__=='__main__': #when y
    path = sys.argv[1]
    filename = sys.argv[2]
    os.chdir(path)
    file = axographio.read(filename)
    # print 'done reading files'

    data = file.data

    no_measurements = len(data)
    measurement_len = len(data[0])

    print(no_measurements)
    print(measurement_len)
    for i, name in enumerate(file.names):
        print(name)
    for i in range(no_measurements):
        for j in range(measurement_len):
            print(data[i][j])
        print('next measurement') 
       #this acts as a seperator when reading the data in python3, anything that cannot be converted to a float works

What this code does is simply take argument from the commad line. (Using the sys module they can be passed to a .py scripts in the command line in the form python2 script.py arg1 arg2 arg2. The output of reading.py is given out using print statements. Note that I print every datum individually in order to avoid truncation of the output. In this way the output is the standard output stdout of the python2 call.

The python3 code is

#analyze.py
import subprocess
import numpy as np

def read_data(path, filename):
    module_name = 'reading.py'
    cmd = 'python2 '+module_name+' '+path+' '+filename
    p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE)
    out, err = p.communicate()
    #out is a bytes object

    result = out.split(b'\n') #the output from different print statements is seperated by "b'\n'" in the bytes object


    #everything below here is concerned with unpacking the output and structuring it as I needed it         
    output = []
    i = 0
    while i < len(result):
        output.append(result[i].decode("utf-8"))
        i+=1
    ind = 0 #this should keep track of where in the output list we are
    no_measurements = int(output[ind])
    ind += 1
    measurement_len = int(output[ind])
    ind += 1

    names = []
    for i in np.arange(ind,ind+no_measurements):
        names.append(output[i])
        ind += 1

    data = []
    measurement = []
    while ind<len(output):
        try:
            measurement.append(float(output[ind]))
            ind+=1
        except:
            data.append(measurement)
            measurement = []
            ind+=1
    data = np.array(data)

    return names, data

What this does is to use the subprocess module to execute as a shell command python2 reading.py path filename. As mentioned above the stdout of this call is the output of its print statements. These are bytes objects encoded with 'UTF-8'. Using the decode method they can be converted to string objects which can then be changed in type using functions like float. (This is another reason why it is useful to print everything seperately since it is rather annoying to have to scan through a string to find arrays or numbers.) In readying.py data are seperated explicitely while being printed which makes it easy to structure them once they are read.

Tags: axographio in python3--read axograph files python3

Nikolai
  • 21
  • 2