I have a string which I read from a file. I need to get the differences of the consecutive values of the string

Question

Code:

import pandas as pd
from collections import defaultdict
import matplotlib.pyplot as plt

def main():
  with open(r'C:\Users\sarad\ss3.dat.log','r') as line: 
    table = defaultdict(dict)
    for line in line:
      if line:
        entry = line.strip()
        if ':' in entry:
            t = entry
        else:
             if line.startswith("Sensor"):
               _, sample,data = entry.split()
               table[t].update({sample:float(data)})
               print(type(data),data)
    print(table)

    df=pd.DataFrame(table).T
    print(df)

    df.plot()
    plt.xticks(
      ticks=range(len(df)),
      labels=df.index,
      rotation=45
    )
    plt.locator_params(axis='x',nbins=10)
    plt.xlabel('Time')
    plt.ylabel('Temperature')
    plt.title('Temperature vs Time ')
    plt.show()
    
main()

ss3.dat.log file is like

00:00:08
Sensor A 76.202574
Sensor B 76.199620
Sensor C 76.092497
Sensor D 76.120303
Sensor E 76.041031
Sensor F 77.463662
Sensor G 85.509276
Sensor H 112.226697
00:00:39
Sensor A 76.201118
Sensor B 76.199382
Sensor C 76.089885
Sensor D 76.118796
Sensor E 76.041040
Sensor F 77.464466
Sensor G 85.509370
Sensor H 112.226071
00:01:10
Sensor A 76.201070
Sensor B 76.193708
Sensor C 76.087995
Sensor D 76.119122
Sensor E 76.041150
Sensor F 77.464949
Sensor G 85.508649
Sensor H 112.225700
00:01:41
Sensor A 76.201223
Sensor B 76.199162
Sensor C 76.089856
Sensor D 76.118216
Sensor E 76.041759
Sensor F 77.465946
Sensor G 85.509358
Sensor H 112.225497
00:02:13
Sensor A 76.199548
Sensor B 76.198596
Sensor C 76.092475
Sensor D 76.121014
Sensor E 76.041773
Sensor F 77.466750
Sensor G 85.509600
Sensor H 112.225685
00:02:44

and so on.

Now I need to do a plot of the temperature gradient. Those float values are the temperatures. And 00:00:08 these values are the clock time. I need to find out the graph of (variation of temparature/ variation of time) with the temprature.(dT/dt vs time)

Your code is not visible. Have you tried to use `str.split(" ")`? With that you have a list of numbers as strings, that can be converted to floats, with which you can calculate your differences — destructioneer, Jul 13 '23 at 09:16
https://stackoverflow.com/questions/743806/how-do-i-split-a-string-into-a-list-of-words and https://stackoverflow.com/questions/379906/how-do-i-parse-a-string-to-a-float-or-int?rq=2 — Marco F., Jul 13 '23 at 09:16

mozway · Answer 1 · 2023-07-13T11:15:09.003

Based on your updated question, I'd probably parse the file manually to create a rectangular DataFrame, then use diff:

import re

out = {}
with open('ss3.dat.log') as f:
    for line in f:
        line = line.strip()
        if re.match(r'\d{2}:\d{2}:\d{2}', line):
            key = line
            out.setdefault(key, {})
        elif (m:=re.search(r'(.*) (\d+.?\d*)', line)):
            out[key][m.group(1)] = float(m.group(2))

df = pd.DataFrame.from_dict(out, orient='index')

print(df)

           Sensor A   Sensor B   Sensor C   Sensor D   Sensor E   Sensor F   Sensor G    Sensor H
00:00:08  76.202574  76.199620  76.092497  76.120303  76.041031  77.463662  85.509276  112.226697
00:00:39  76.201118  76.199382  76.089885  76.118796  76.041040  77.464466  85.509370  112.226071
00:01:10  76.201070  76.193708  76.087995  76.119122  76.041150  77.464949  85.508649  112.225700
00:01:41  76.201223  76.199162  76.089856  76.118216  76.041759  77.465946  85.509358  112.225497
00:02:13  76.199548  76.198596  76.092475  76.121014  76.041773  77.466750  85.509600  112.225685

print(df.diff().div(pd.to_timedelta(df.index).total_seconds(), axis=0))

              Sensor A  Sensor B  Sensor C  Sensor D      Sensor E  Sensor F  Sensor G  Sensor H
00:00:08           NaN       NaN       NaN       NaN           NaN       NaN       NaN       NaN
00:00:39 -3.733333e-05 -0.000006 -0.000067 -0.000039  2.307692e-07  0.000021  0.000002 -0.000016
00:01:10 -6.857143e-07 -0.000081 -0.000027  0.000005  1.571429e-06  0.000007 -0.000010 -0.000005
00:01:41  1.514851e-06  0.000054  0.000018 -0.000009  6.029703e-06  0.000010  0.000007 -0.000002
00:02:13 -1.259398e-05 -0.000004  0.000020  0.000021  1.052632e-07  0.000006  0.000002  0.000001

df.diff().div(pd.to_timedelta(df.index).total_seconds(), axis=0).plot()

Output:

original answer

I would use numpy's fromstring and diff:

import numpy as np

s = '77.65 77.90 78.20 78.30 79.60 80.40'

out = np.diff(np.fromstring(s, sep=' ')).tolist()

Output:

[0.25, 0.3 , 0.1 , 1.3 , 0.8 ]

''' with open(r'C:\Users\sarad\ss4.dat.log','r') as line: table = defaultdict(dict) for line in line: if line: entry = line.strip() if ':' in entry: t = entry else: if line.startswith("Sensor"): _, sample,data = entry.split() table[t].update({sample:float(data)}) out=np.diff(np.fromstring(data,sep=' ')).tolist() print(out) ''' — Saradindu Samanta, Jul 13 '23 at 09:41
I attached the code . The differences are coming like [][][][] that . There is no number in it. — Saradindu Samanta, Jul 13 '23 at 09:41
Here, `data` doesn't seem to be a string. What is the output of `print(type(data), data)`? Also please post this code as [edit](https://stackoverflow.com/posts/76677713/edit) to your question — mozway, Jul 13 '23 at 09:46
Let's start from the beginning, please [edit](https://stackoverflow.com/posts/76677713/edit) your question and provide the full content of your file. You didn't tell use the whole story here ;) — mozway, Jul 13 '23 at 09:56

score 0 · Answer 2 · answered Jul 13 '23 at 09:24

In case you don't want to import numpy (which I think is overkill for something so trivial) you could do this:

def subtract(s: str) -> list[float]:
    vals = [float(n) for n in s.split()]
    return [b - a for a, b in zip(vals[1:], vals)]

s = '77.65 77.90 78.20 78.30 79.60 80.40'

print(subtract(s))

I have a string which I read from a file. I need to get the differences of the consecutive values of the string

2 Answers2

original answer