Sometimes, rather than a thousand words, just a couple of images are enough to convey the meaning of certain concepts.
As can be inferred, in the first case, it is an open-loop system, because once the stone is thrown, it is no longer possible to control its trajectory. In the second case, however, the continuous monitoring of the trajectory, with the ability to modify its characteristic curve through control, is defined as a closed-loop system.
In graphical terms of representation for "block diagrams", the difference is represented as follows:
In the first case, an open-loop system can be represented in this way:
In the second case, a closed-loop system can be represented in the following way:
From this, one can deduce a significant complexity in dynamic management compared to the open-loop system. However, without delving into details, it is not difficult to understand that an open-loop system is merely a particular case of a closed-loop system, as easily illustrated by the following diagram.
Sequences – Vectors – Simulations – Solutions
The inspiration comes from the research conducted by Alessandro Delfanti, an associate professor at the Faculty of Information at the University of Toronto, who addresses a topic concerning the synergy between humans and machines in certain work environments.
Delfanti conducted research (in 2017) based on numerous interviews with workers at the first Italian Amazon warehouse. The professor's analysis focused on certain types of tasks that are an integral part of the logistical context of warehouse management.
For this article, we consider the work of the "Stower": it is important to note that the goods we order online from Amazon are stored chaotically.
The process followed by warehouse workers consists of four steps, starting with receiving the goods, moving to storage (stow), picking, and finally packing for shipment.
I know this because I have also passed the procedures related to the hiring criteria for working in Amazon warehouses.
The worker at the second step of the work process, the so-called "Stower", is responsible for taking the goods from a "Tote" container and placing them inside a "Bin" cell of the tower, or "Pod" cabinet, in the assigned area.
Inside the Tote, the goods are placed randomly, with various weights and sizes ("a confused organized arrangement", as they say).
The Bins are containers arranged within the Pod and come in six different sizes, depending on the dimensions of the objects to be stored.
An optical device indicates to the Stower where the item taken from the Tote should be placed.
The Stower has discretion in picking the goods and does so based on the availability of placing the item in one of the free Bins.
At the time of "login", at the beginning of the shift, the system presents a series of "question tests" to the operator regarding their psychophysical state.
Obviously, the "operational sequences" are constantly monitored, as the IT system is the "core" of the warehouse.
The barcode reader, in addition to providing
practical indications for the placement of goods and thus "performing
management activities", obviously, the monitoring
algorithm, sends feedback on the worker's performance.
According to the workers, the tool is not very precise in indicating the optimal placement of objects, and "it happens that the goods are not in the closest point to the picker, effectively forcing the worker to maintain a very high pace and move continuously", Delfanti explained in his research.
When the worker finishes their shift, the system dismisses the operator with courteous phrases, but, as far as I know, it does not ask the worker if their level of "stress" has changed concerning the work performed.
From the above, I was inspired to write this article, which aims to serve as an educational example for using Python and auxiliary libraries related to deep learning and machine learning.
TensorFlow: It is considered one of the best Python libraries for deep learning applications. Developed by the Google Brain Team, it offers a wide range of flexible tools, instruments, and Open Source community resources.
Keras: The Python library closely related to TensorFlow, used for deep learning tasks, allows for rapid testing of deep neural networks. Keras provides the necessary tools to build models, visualize graphs, and analyze datasets.
Scikit-learn: It is a machine learning library that contains algorithms for classification, regression, and clustering, including support vector machines, logistic regression, and Bayesian classifiers.
NumPy: It is the library used for processing large multidimensional arrays and matrices. It is based on a comprehensive set of high-level mathematical functions, making it particularly useful for fundamental scientific calculations in deep learning techniques.
Pandas: One of the open-source Python libraries primarily used in data science and deep learning, this library provides tools for data manipulation and analysis.
Matplotlib: A library for creating plots, used to generate a wide range of static, animated, and interactive visualizations, such as line charts, scatter plots, bar charts, histograms, and more.
Csv: A Python library for handling "Comma Separated Values" files. It is one of the most widely used formats, especially in scientific fields or when dealing with large amounts of data. It is a storage method for structured data in spreadsheet-style tabular format.
Roughly speaking, the structure of this educational project has been conceived in this way:
Structure of the Educational Project
Software Code Snippets in Python:
The Python language is quite readable, but in this case, given the complexity of the project, only the most significant parts of the various subroutines (functions) will be presented. These were used for the experimental implementation of three typical applications utilizing predictive deep learning systems.
Sequence Generators
Essentially, these are four functions that generate sequences based on the list, starting from the specifications introduced by Python's "Random" library:
work_type = [("wa",1),("wb",2),("wc",3),("wd",4),("we",5),("wf",6) ]
It is a list of tuples containing an identifying string for the type of objects and a numerical value that could indicate a level of "complexity related to the storage function", in this purely educational case.
To generate the sequences, it is sufficient to import Python's Random library:
mport random
To perform a functionality test, just a few lines of code are sufficient:
work_type = [("wa",1),("wb",2),("wc",3),("wd",4),("we",5),("wf",6) ]
l=len(work_type)
print("lunghezza del vettore", l)
wr = random.randint(0, (l-1))
print("wr", wr, "work_type", work_type[wr])
print ("work_type", work_type[wr][1])
wtr=work_type[wr]
print ("wtr", wtr)
A possible output:
len of vector is 6
wr 2 work_type ('wc', 3)
work_type 3
wtr ('wc', 3)
The standardized sequence generation routine could be created using code similar to the one shown below, where a variable is defined to indicate the number of sequences to generate.
Code for Standard Sequence:
def standard_s():
number_s = 360 # 360
nw= 0
T=0
nwa=0
nwb=0
nwc=0
nwd=0
nwe=0
nwf=0
yO=[]
x=[]
...
while nw < number_s:
nw=nw+1
print("number sequence",nw)
wr = random.randint(0, (l-1))
wtr=work_type[wr]
print ("wtr=",wtr)
T=work_type[wr][1] # value
O=work_type[wr][0] # index
print("O=", O, "T=", T )
x.append(nw)
yO.append(T)
if O == "wa":
nwa=nwa+1
if O == "wb":
nwb=nwb+1
if O == "wc":
nwc=nwc+1
if O == "wd":
nwd=nwd+1
if O == "we":
nwe=nwe+1
if O == "wf":
nwf=nwf+1
print("seq_type", "wa=",nwa, "wb=", nwb, "wc=", nwc, "wd=",nwd, "we=",nwe, "wf=", nwf)
# Creazione del grafico
plt.bar(x, yO)
plt.title("Grafico a barre sequence standard")
plt.xlabel("Nuber sequences")
plt.ylabel("operation type")
plt.show()
return x, yO
output to terminal:
...
number sequence 359
wtr ('wb', 2)
O wb T= 2
number sequence 360
wtr ('wd', 4)
O wd T= 4
sequence_type wa= 61 wb= 58 wc= 59 wd= 68 we= 57 wf 57
Graphic output:
As you can see, the total number for each type of material is distributed almost evenly:sequence_type wa= 61 wb= 58 wc= 59 wd= 68 we= 57 wf 57.
The choice of a length of 360 elements was also made for a visualization of this type, which could be imagined as the face of a clock with hands: (Polar chart rotated clockwise):
The graphic output is provided to us by the Matplotlib library, previously called by the instruction:
import matplotlib.pyplot as plt
For generating sequences characterized by a "conditioning" during the generation phase, as might be necessary to obtain a sequence with a predominance of objects characterized by lower "values", it is necessary to adopt some precautions. This ensures that random extraction favors that type of object, for example, by using certain parameters as outlined below:
Code for Light Sequence:
def light_s():
number_s = 360
nw= 0
T=0
Total=0
nwa=0
nwb=0
nwc=0
nwd=0
nwe=0
nwf=0
a=True
b=True
c=True
d=True
e=True
f=True
maxwa = 120 #48
maxwb = 80 #35
maxwc = 75 #26
maxwd = 50 #14
maxwe = 25 #7
maxwf = 15 #5
iup=0
sum_max = maxwa + maxwb + maxwc + maxwd + maxwe + maxwf
while nw < number_s:
nw=nw+1
print("number of work",nw)
wr = random.randint(0, (l-1))
T=work_type[wr][1]
O=work_type[wr][0]
print("O=", O, "T=", T )
if O == "wa":
maxwa = maxwa-1
if maxwa > 0:
nwa=nwa+1
x.append(nw)
yO.append(T)
if maxwa <= 0:
nw=nw-1
a = False
print ("a=", a)
i f O == "wb":
if maxwb > 0:
nwb=nwb+1
maxwb = maxwb-1
x.append(nw)
yO.append(T)
if maxwb <= 0:
nw=nw-1
b = False
print ("b=", b)
if O == "wc":
if maxwc > 0:
nwc=nwc+1
maxwc = maxwc-1
x.append(nw)
yO.append(T)
if maxwc <= 0:
nw=nw-1
c = False
print ("c=", c)
if O == "wd":
if maxwd > 0:
nwd=nwd+1
maxwd = maxwd-1
x.append(nw)
yO.append(T)
if maxwd <= 0:
nw=nw-1
d = False
print ("d=", d)
if O == "we":
if maxwe > 0:
nwe=nwe+1
maxwe = maxwe-1
x.append(nw)
yO.append(T)
if maxwe <= 0:
nw=nw-1
e = False
print ("e=", e)
if O == "wf":
if maxwf > 0:
nwf=nwf+1
maxwf = maxwf-1
x.append(nw)
yO.append(T)
if maxwf <= 0:
nw=nw-1
f = False
print ("f=", f)
if (a==False and b==False and c==False and d==False and e==False and f==False) and nw < number_s:
print("maxwa", maxwa)
print("maxwb", maxwb)
print("maxwc", maxwc)
print("maxwd", maxwd)
print("maxwe", maxwe)
print("maxwf", maxwf)
maxwa += 1
maxwb += 1
maxwc += 1
iup +=1
print ("increments update",)
nw= nwa+nwb+nwc+nwd+nwe+nwf
print ("increment update counter", iup)
print ("nw=", nw)
print ("sum max", sum_max, nw)
print("operations", "wa=",nwa, "wb=", nwb, "wc=", nwc, "wd=",nwd, "we=",nwe, "wf", nwf)
# Creazione del grafico
plt.bar(x, yO)
plt.title("Grafico a barre sequence light")
plt.xlabel("Nuber sequences")
plt.ylabel("operation type")
plt.show()
#--------------
return x, yO
#-----------------------------
output to terminal:
...
number of work 360
O= wc T= 3
c= False
increments update
increment update counter 2
nw= 366
sum max 365 366
operations wa= 119 wb= 81 wc= 76 wd= 50 we= 25 wf 15
Bar graph output:
Polar graph (rotated clockwise):
As can be verified from reading the code, at a certain point, only the extractions for objects with lower values are favored.
With very similar precautions, one can generate a sequence that instead tends to favor objects with higher values, as indicated by the code summary provided below.
Code Summary for Hard Sequence:
def hard_s():
number_s = 360
nw= 0
T=0
Total=0
nwa=0
nwb=0
nwc=0
nwd=0
nwe=0
nwf=0
colors=[]
sizes=[]
a_s=True
b_s=True
c_s=True
d_s=True
e_s=True
f_s=True
maxwa = 20
maxwb =30
maxwc = 40
maxwd = 60
maxwe = 80
maxwf = 100
iup=0
sum_max = maxwa + maxwb + maxwc + maxwd + maxwe + maxwf
while len(yh) < number_s:
nw=nw+1
wr = random.randint(0, (l-1))
T=work_type[wr][1]
O=work_type[wr][0]
if O == "wa":
maxwa = maxwa-1
if maxwa > 0:
nwa=nwa+1
xh.append(nw)
yh.append(T)
if maxwa <= 0:
nw=nw-1
a_s = False
….
if O == "wf":
if maxwf > 0:
nwf=nwf+1
maxwf = maxwf-1
xh.append(nw)
yh.append(T)
if maxwf <= 0:
nw=nw-1
f_s = False
if (a_s==False and b_s==False and c_s==False and d_s==False and e_s==False and f_s==False) and nw < number_s:
maxwd += 1
maxwe += 1
maxwf += 1
iup +=1
nw= nwa+nwb+nwc+nwd+nwe+nwf
print ("increment update counter", iup)
print ("nw=", nw)
# Creazione del grafico
plt.bar(xh, yh)
plt.title("Grafico a barre sequence hard")
plt.xlabel("Number sequences")
plt.ylabel("operation type")
plt.show()
plt.close()
#--------------
print ("lenyO", len(yh))
print ("END HARD SEQUENCE")
x= xh
yO = yh
polar_plotting(x,yO)
return x, yO
graphic output
Polar graph (rotated clockwise):
As can be easily observed, in this case, sequences of "objects" with higher "value" are favored. The term value could be associated, for example, with a level of stress attributable to the storage operation, whether because the object is heavy or because it needs to be stored in an inconvenient position, perhaps high up in the "Pod", using a ladder.
Finally, I introduced a variant to generate a type of random sequence, but starting from the type of objects. In this way, one can unbalance or not a typological series and consequently the entire sequence generation.
“Casual” sequence code:
def casual_s():
number_s = 360
nw= 0
T=0
Total=0
nwa=0
nwb=0
nwc=0
nwd=0
nwe=0
nwf=0
#yc=[]
#xc=[]
colors=[]
sizes=[]
a_s=True
b_s=True
c_s=True
d_s=True
e_s=True
f_s=True
max =[]
for m in range(6):
r = random.randint(10, 100)
max.append(r)
maxwa = max[0]
maxwb = max[1]
maxwc = max[2]
maxwd = max[3]
maxwe = max[4]
maxwf = max[5]
iup=0
sum_max = maxwa + maxwb + maxwc + maxwd + maxwe + maxwf
print ("sum max", sum_max)
while len(yc) < number_s:
nw=nw+1
wr = random.randint(0, (l-1))
T=work_type[wr][1]
O=work_type[wr][0]
if O == "wa":
maxwa = maxwa-1
if maxwa > 0:
nwa=nwa+1
xc.append(nw)
yc.append(T)
if maxwa <= 0:
nw=nw-1
a_s = False
….
if O == "wf":
if maxwf > 0:
nwf=nwf+1
maxwf = maxwf-1
xc.append(nw)
yc.append(T)
if maxwf <= 0:
nw=nw-1
f_s = False
if (a_s==False and b_s==False and c_s==False and d_s==False and e_s==False and f_s==False) and nw < number_s:
maxwa += 1
maxwb += 1
maxwc += 1
maxwd += 1
maxwe += 1
maxwf += 1
iup +=1
print ("increments update", iup)
nw= nwa+nwb+nwc+nwd+nwe+nwf
print ("increment update counter", iup)
print("operations CASUAL SEQUENCE", "wa=",nwa, "wb=", nwb, "wc=", nwc, "wd=",nwd, "we=",nwe, "wf", nwf)
# Creazione del grafico
plt.bar(xc, yc)
plt.title("Grafico a barre sequence casual")
plt.xlabel("Number sequences")
plt.ylabel("operation type")
plt.show()
plt.close()
print("len yc",len(yc))
print ("END CASUAL SEQUENCE")
polar_plotting(xc, yc)
return xc, yc
#-----------------------------
Terminal output:
wr 0 work_type ('wa', 1)
1
sum max 259
increments update 111
increment update counter 111
operations CASUAL SEQUENCE wa= 50 wb= 39 wc= 47 wd= 103 we= 49 wf 72
graphic output
Polar graph (rotated clockwise):
As you can see, in this case, a sequential anomaly is presented in the 180°-270° quadrant, also reported on the terminal:
operations CASUAL SEQUENCE wa= 50 wb= 39 wc= 47 wd= 103 we= 49 wf= 72
Now, let's suppose we analyze the sequences with a criterion to determine their stress level based, for example, on the fact that high-value objects to be stored appear consecutively, as indicated by the following lines of code:
Code summary for the analysis phase:
…
event = False
for i in range(1,len(yO),1):
if i < (len(yO)-step):
s= yO[i:i+step]
if event == True:
j +=1
if j >= 2:
j=0
event = False
stress = stress -1
if type[key] == 6 or type[key] == 5:
stress = stress +1
#-----------------------------------------
for key in range (len(type)):
seq = s.count(type[key])
if seq >= (step-1):
stress +=1
event = True
break
print ("lo stress dell'ultima sequenza è ", stress)
return stress
If, while examining (scanning) a sequence, it is noted that there are successions of objects belonging to the two highest difficulty levels for storage within a certain control range, then a stress level counter is incremented. This way, one could record situations of this kind for hard sequences, for example:
terminal output:
sequence_type wa= 63 wb= 50 wc= 54 wd= 66 we= 60 wf= 67
START ANALYSIS
increment update counter 63
nw= 360
lenyO 360
END HARD SEQUENCE
lo stress dell'ultima sequenza è 115
Whereas for a casual sequence, for example, a situation like the one below might occur:
terminal output:
sequence_type wa= 58 wb= 59 wc= 49 wd= 65 we= 66 wf= 63
START ANALYSIS
sum max 469
increments update 0
nw= 360
len yO 360
END CASUAL SEQUENCE
The stress of the last sequence is 38.
As you can see, at first glance, the sequences appear similar in numerical terms, but they are completely different in terms of stress values:
Hard Sequence: sequence_type wa= 63 wb= 50 wc= 54 wd= 66 we= 60 wf= 67 - stress=115
Casual Sequence: sequence_type wa= 58 wb= 59 wc= 49 wd= 65 we= 66 wf= 63 - stress= 38
At this point, it might be interesting to see how one or more analysis systems based on deep learning techniques might behave. To do this, sequences need to be generated to "train" the model and other sequences to "test" the model you want to use.
The goal, therefore, is to build a dataset containing a number of vectors, associating them with their respective stress levels, both for training and testing. This can be done by writing files such as "Train_set.csv" and "Test_set.csv."
Code for generating the “Train_set.csv” and the “Test_set.csv”:
def train_gen():
print("START TRAIN GENERATOR")
global yO, stress, v_a, mv
dimension_train= 1000
#dv=[]
#ds=[]
for d in range (dimension_train):
#print ("d",d)
vectors()
analysis()
#print("stress=", stress)
dv.append(yO)
ds.append(stress)
yO=[]
rwd=[]
rt=[]
for rw in range (dimension_train):
rwd=dv[rw]
rwd.append(ds[rw])
print ("rwd: ",rwd)
rt.append(rwd)
with open('Train_seq_num.csv', 'w', newline='') as csvfile:
rw_writer = csv.writer(csvfile)
for rw in range (dimension_train):
rw_writer.writerow(rt[rw])
v=[]
for c in range(len(dv)):
#print ("len a(c)",len(dv[c]))
v.append(len(dv[c]))
#print ("lunghezze vettori",v)
mv= max(v)
#print("lunghezza massima", mv)
#v_a=[]
for p in range (len(dv)):
x=max(v)-v[p]
#print("x",x)
vx=dv[p]
for n in range (x):
vx.append(0)
#print("vx",vx)
v_a.append(vx)
...
return v_a, ds, mv
#------------------
This function generates a file whose title is “Train_seq_num.csv”, containing a thousand vectors:
dimension_train= 1000
of length 360 values relating to the level of difficulty inherent to the type of material, plus the associated stress value: rwd.append(ds[rw]) positioned in the last box of the vector.
The vectors are generated in relation to random sequences determined by the “vectors()” function:
def vectors():
global yO
# vectors generator
vg = 1
for counter in range (vg):
v = random.randint(1, 2)
if v == 1:
hard_s()
if v == 2:
casual_s()
print ("vector N.", counter)
In graphic terms, the question could therefore be summarized in this way:
The same goes for generating the test file “Test_seq_num.csv”, which in graphic terms could look something like this:
At this point, the data is available to test some deep learning models to see what prediction results can be obtained.
To do this, there is a need to access the two files to extract the training and test data. This task is delegated to Python's Pandas library, which accomplishes it through the following lines of code:
Data extraction with Pandas (and processing with Numpy
import pandas as pd
import numpy as np
...
train_set = pd.read_csv("Train_seq_num.csv", header = None)
print("train_set ",train_set.head(3))
print("len columns ",len(train_set.columns))
print("last columns", train_set[360]) # stress vector
trv_s = np.array(train_set[360])
vtr_g=np.array(train_set)
print("len vtrg",len(vtr_g))
vtr= vtr_g[0]
vtr = vtr[:-1]
#--------------------------------
print("len_vtr=",len(vtr))
print("len_stress_vector=",len(trv_s))
#print("primo vettore", vtr)
vtr= vtr_g[len(trv_s)-1]
vtr = vtr[:-1]
#print("ultimo vettore", vtr)
tr_v=[]
for tr in range (len(trv_s)):
vtr= vtr_g[tr]
vtr = vtr[:-1]
tr_v.append(vtr)
print("len_tr_v", len(tr_v))
#--------------------------
X_train = tr_v
y_train = trv_s
#--------------------------
# datset test extaction
test_set = pd.read_csv("Test_seq_num.csv", header = None)
print("test_set ",test_set.head())
print("len columns ",len(test_set.columns))
print("last columns", test_set[360]) # stress vector
tev_s = np.array(test_set[360])
#print ("train_vector_stress", train_vector_stress)
vte_g=np.array(test_set)
print("len vte_g",len(vte_g))
vte= vte_g[0]
vte = vte[:-1]
#------
print("len_vte=",len(vte))
print("len_stress_v_test=",len(tev_s))
#print("primo vettore", vtr)
vte= vte_g[len(tev_s)-1]
vte = vte[:-1]
#print("ultimo vettore", vtr)
te_v=[]
for te in range (len(tev_s)):
vte= vte_g[te]
vte= vte[:-1]
te_v.append(vte)
print("len_te_v", len(te_v))
#--------------------------
X_test = te_v
y_test = tev_s
#*********************************
In particular, it is important to note the following: the files contain rows with the length of the stored records: 360+1 (starting from "0").
This can be observed with the print request of the "head": print("train_set ",train_set.head(3))
train_set 0 1 2 3 4 5 6 7 8 ... 352 353 354 355 356 357 358 359 360
0 3 1 2 5 3 3 1 3 3 ... 5 4 6 6 6 5 5 6 130
1 5 3 1 5 2 3 2 5 5 ... 4 6 6 6 6 4 4 6 120
2 5 5 4 6 1 3 1 4 1 ... 4 4 6 6 5 5 6 5 118
3 6 6 1 1 6 6 4 5 5 ... 2 6 2 2 4 1 5 5 36
4 2 3 2 4 4 3 3 5 6 ... 5 4 6 4 2 6 2 5 82
[5 rows x 361 columns]
In particular, the last column contains the "stress" value, associated with the vector:
print("last columns", train_set[360])
last columns 0 130
1 120
2 118
3 36
4 82
...
995 36
996 26
997 131
998 58
999 142
Name: 360, Length: 1000, dtype: int64
It is essential, however, to divide the input data (the vectors containing the sequences with the difficulty level values of the objects) from the "stress" data, which are arbitrarily calculated using the previously discussed criterion, in order to obtain two lists:
X_train (input vectors for the training models)
y_train (stress levels associated with each vector sequence for model training)
Similarly, for testing the model, the vectors need to be extracted:
X_test (input vectors for the test models)
y_test (stress levels associated with each vector sequence for model testing)
The Numpy library is used precisely for this purpose.
Therefore, for example, in this case, the y_test vector will have the following characteristic:
y_test = tev_s
print ("te_s", tev_s)
te_s [ 37 112 42 124 134 132 117 62 134 124]
What we can expect from the predictions of the deep learning models is something that closely approximates these values!
Sequential + LSTM Model
As initially mentioned, there can be various models that could be tested to verify their efficiency in this context, which might seem simple to some, but is nevertheless not trivial.
One of the most widely used libraries in this field is "TensorFlow", whose peculiarity is that it works well with CPUs but has excellent performance with GPUs (high-performance graphics processing units). However, its ideal environment is TPUs: tensor processing units.
TensorFlow is by far the most used AI engine, also because it is open source and is supported worldwide by a vast community.
TensorFlow combines various machine learning and deep learning (or neural networking) models and algorithms and makes them accessible through a common interface.
Keras is also an open-source library for high-level neural networks, designed to be easy to use, modular, and easily extensible. In particular, this library is written in Python, and its main "supporter" is TensorFlow.
With TensorFlow and Keras, one can configure neural networks, and it is precisely with these tools that I will try to provide models, for educational purposes, to attempt to generate predictions.
One of these models could be the one that refers to LSTM (long-short term memory), which has an internal structure capable of propagating information through long sequences.
I must clarify from the outset that models can be constructed with various systems; those proposed are just some that, in addition to referring to a layer of the network specifically modeled for the use of LSTM, also refer to other layers that contribute to determining the behavior of the Deep Learning Network.
In particular, in the proposed model, a first input layer will be used, which refers to the "Sequential" function, while the penultimate layer refers to an activation function called "relu", which graphically presents itself in this way:
The last layer, finally, will have a single neuron that will produce a value, the prediction, related to the repeated processing of input values for a number of times that can be predetermined or determined based on parameters related to a performance estimate.
In simple terms, the first model that can be experimented with to perform a stress prediction, given a series of training sequences on the data, is as follows:
model = Sequential()
model.add(tf.keras.Input(shape=(360, 1)))
model.add(LSTM(64))
#model.add(Dense(8, activation="sigmoid"))
model.add(Dense(8, activation="relu"))
model.add(Dense(1))
Once the model is chosen, it is necessary to "pass to it" the working parameters that determine its behavior:
model.compile(loss='mean_squared_error', optimizer='adam', metrics=['mae'])
In this case, the model will use the "loss" parameter, which is a function used to determine the error between the value the model produces as the training sequences (epochs) are implemented and the "target" value contained in the y_train vector for each input vector. In this case, the formula is as follows:
loss
= mean(square(y_true – y_pred))
The
control metric adopted to visualize the error in this case has been
used in relation to the following formula:
keras.metrics.MeanAbsoluteError(name="mean_absolute_error",
dtype=None)
which
refers to the following formula: metrics
= mean(abs(y_true - y_pred))
Thus,
the model is "launched" to perform training on the data,
storing its parameters in a variable:
history=model.fit(X_train,
y_train,
epochs=100,
batch_size=36)
where the "batch_size" value defines the number of samples that will be propagated through the network.
So, in this case, having 360 training samples and a batch_size of 36, the algorithm takes the first 36 samples (from the 1st to the 36th) from the training dataset and models the network. Thus, in this case, it sifts through all the input data contained in X_train 100 times, in blocks of 36.
The results can be displayed graphically using the following lines of code:
plt.figure(figsize=(10, 6))
plt.plot(history.history['mae'], label='mae')
plt.plot(history.history['loss'], label='loss')
plt.legend()
plt.show()
which is equivalent to obtaining, for example, something similar to this:
Once the training phase is completed, it is possible to move on to the validation phase using the following line of code:
model.evaluate(X_test,
y_test,
batch_size=6)
In this case, a batch_size of 6 was used on the input data X_test.
The result obtained can also be displayed graphically using the following lines of code:
# Creazione del grafico
fig, ax = plt.subplots(figsize=(10, 6.10), layout='constrained')
ax.plot(y_test, color="green", marker="s", label='real_test', linestyle="")
ax.plot(v_p, color="red", marker="*",label='prediction',linestyle="")
ax.plot(e, color="gray", marker="*",label='error %',linestyle="-")
ax.set_xlabel('x label') # Add an x-label to the Axes.
ax.set_ylabel('y label')
ax.legend(bbox_to_anchor=(0., 1.05, 1., .102), loc='lower left',
ncols=2, mode="expand", borderaxespad=0.)# Add a legend.
plt.title("Grafico stress events")
plt.xlabel("Number sequences")
plt.ylabel("stress events")
#plt.show(block=False)
plt.show()
#plt.pause(100)
plt.close()
obtaining, in this case, a result of this kind:
As can be observed, the plotting pertains to vectors that have been derived in relation to the data produced by the model, which essentially processed the training model 100 times and the test model a few times, as reported by the following information copied from the terminal:
….
Epoch 80/100
28/28 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step - loss: 188.0179 - mae: 10.4328
Epoch 81/100
28/28 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step - loss: 172.8304 - mae: 10.3376
Epoch 82/100
28/28 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step - loss: 193.7802 - mae: 10.8849
Epoch 83/100
28/28 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step - loss: 167.6130 - mae: 9.9958
Epoch 84/100
28/28 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step - loss: 168.5521 - mae: 10.0910
Epoch 85/100
28/28 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step - loss: 166.1593 - mae: 9.9478
Epoch 86/100
28/28 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step - loss: 181.2276 - mae: 10.3700
Epoch 87/100
28/28 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step - loss: 167.1966 - mae: 10.0520
Epoch 88/100
28/28 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step - loss: 164.8326 - mae: 10.2739
Epoch 89/100
28/28 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step - loss: 152.0353 - mae: 9.4940
Epoch 90/100
28/28 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step - loss: 197.9228 - mae: 10.6248
Epoch 91/100
28/28 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step - loss: 159.9308 - mae: 9.9471
Epoch 92/100
28/28 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step - loss: 147.8240 - mae: 9.1989
Epoch 93/100
28/28 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step - loss: 143.6818 - mae: 9.3541
Epoch 94/100
28/28 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step - loss: 148.8043 - mae: 9.3832
Epoch 95/100
28/28 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step - loss: 167.6693 - mae: 10.0714
Epoch 96/100
28/28 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step - loss: 152.4915 - mae: 9.4687
Epoch 97/100
28/28 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step - loss: 150.1740 - mae: 9.5651
Epoch 98/100
28/28 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step - loss: 147.2422 - mae: 9.5501
Epoch 99/100
28/28 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step - loss: 133.1573 - mae: 8.8903
Epoch 100/100
28/28 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step - loss: 142.7763 - mae: 9.1165
end
2/2 ━━━━━━━━━━━━━━━━━━━━ 0s 18ms/step - loss: 78.7232 - mae: 7.1506
The input vector is a vector processed by Numpy:
X_test [[2 3 3 ... 3 5 3]
[3 4 2 ... 5 4 6]
[2 1 5 ... 2 2 2]
...
[3 1 1 ... 3 5 6]
[4 1 1 ... 5 5 4]
[6 2 6 ... 5 4 5]]
The prediction vector is composed of floating values:
rough vector [[ 47.26421 ]
[127.515976]
[ 33.683308]
[128.81313 ]
[130.34879 ]
[131.35663 ]
[120.9835 ]
[ 58.3758 ]
[131.32555 ]
[106.2321 ]]
which have been normalized to integer values using the function:
p = math.floor(y_pred[y_p])
In the procedure of conversion and reconstruction of the vectors, in order to derive a series of "delta" values indicating a percentage error related to the predicted value, for each calculated value belonging to the y_test vector, the following lines of code are provided:
v_p=[]
e=[]
for y_p in range(len(y_pred)):
p = math.floor(y_pred[y_p])
v_p.append(p)
#y_pred =math.floor(y_pred)
print ("y_test", y_test)
print ("vettore di predizione",v_p)
for d in range(len(y_test)):
delta = ((y_test[d] - v_p[d])/y_test[d])*100
delta= int(delta)
e.append(delta)
print ("vettore delta",e)
The result obtained in this case is the following:
y_test [ 37 112 42 124 134 132 117 62 134 124]
prediction vector[47, 127, 33, 128, 130, 131, 120, 58, 131, 106]
delta vector [-27, -13, 21, -3, 2, 0, -2, 6, 2, 14]
A possible alternative use inherent to this type of model is the following:
# model = tf.keras.models.Sequential([
# tf.keras.layers.Lambda(lambda x: tf.expand_dims(x, axis=-1),
# input_shape=[None]),
# #tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(1024, return_sequences=True)),
# tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(512, return_sequences=True)),
# tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(256, return_sequences=True)),
# tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(128, return_sequences=True)),
# tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(64)),
# tf.keras.layers.Dense(1),
# ])
#----------------------------------------
# model.summary()
but it is definitely more demanding on the GPU and requires more processing time.
Linear Regression Model
Another model worth testing in this case is a predictor based on the statistical concept of linear regression, which represents a method for estimating the expected conditional value of a dependent variable Y, given the values of other independent variables X 1 , … , X k : E [ Y | X 1 , … , X k ].
In this case, a variant of this model will be used: multiple linear regression (MLR), also known simply as multiple regression.
This is a statistical technique that uses several explanatory variables to predict the outcome of a response variable.
The goal of MLR is to model the linear relationship between the explanatory (independent) variables and the response (dependent) variables.
In essence, multiple regression is an extension of ordinary least squares (OLS) regression because it involves more than one explanatory variable.
The formula for calculating multiple linear regression (MLR) is as follows:
yi=β0+β1*i1+β2*i2+...+βp*ip+ϵ
where for i = n observations:
yi = dependent variable
xi = explanatory variables
β0 = y-intercept (constant term)
βp = slope coefficients for each variable
ϵ = model error term (also known as residuals)
The MLR technique assumes that there is a linear relationship between the dependent and independent variables, that the independent variables are not highly correlated, and that the variance of the residuals is constant.
The coefficient of determination (R-squared) is a statistical metric used to measure the amount of variation in the outcome that can be explained by the variation in the independent variables.
R² always increases as more predictors are added to the MLR model, even if the predictors may not be related to the outcome variable.
This model is used to determine a mathematical relationship between different random variables.
In other words, MLR examines how multiple independent variables are related to a dependent variable.
Once each of the independent factors has been determined to predict the dependent variable, information on the multiple variables can be used to create an accurate forecast of the level of effect they have on the outcome variable.
The model creates a relationship in the form of a straight line (linear) that best fits all individual data points; however, this indeed limits its accuracy in prediction for the case we want to subject it to, if you think about it!
Nevertheless, let's see the (simpler) constitution of such a model:
def LR():
print("START LINEAR REGRESSION")
global X_train, y_train, X_test, y_test
import numpy as np
import math
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, r2_score
## Creiamo un semplice dataset
# train_gen()
X= X_train
y = y_train
#y=ds
print("*************************")
#print ("X=", X)
#print("***")
#print("stress array di train", y)
# # Inizializziamo e addestriamo il modello
model_multi = LinearRegression()
model_multi.fit(X, y)
y_pred = model_multi.predict(X)
mse = mean_squared_error(y, y_pred)
r2 = r2_score(y, y_pred)
print(f'MSE: {mse}')
print(f'R-squared: {r2}')
print("-----------------")
# # # Otteniamo i coefficienti
intercept_multi = model_multi.intercept_
coefficients_multi = model_multi.coef_
print(f'Intercetta (β₀): {intercept_multi}')
print(f'Coefficiente β₁: {coefficients_multi[0]}')
print(f'Coefficiente β₂: {coefficients_multi[1]}')
print("-------------------")
#----------------
n_k =[]
e=[]
num_test = len(y_test)
for n in range(num_test):
X = X_test[n]
print ("X=",X)
print("stress_test=",y_test[n])
k= model_multi.predict([X])
print ("stress presunto=",k)
n_k.append(k)
delta = ((y_test[n] - k)/y_test[n])*100
e.append(delta)
#--------------------
print ("vettore calcolato di test per stress", y_test)
print ("risultato delle predizioni", n_k)
#-----------------------------------------------------
# Creazione del grafico
fig, ax = plt.subplots(figsize=(10, 6.10), layout='constrained')
# #ax.plot(ds, color="blue", marker="o", label= 'train stress', linestyle="")
ax.plot(y_test, color="green", marker="s", label='real_test', linestyle="")
ax.plot(n_k, color="red", marker="*",label='prediction',linestyle="")
ax.plot(k, color="red", marker="*",label='prediction',linestyle="")
ax.plot(e, color="gray", marker="*",label='error %',linestyle="-")
# ## plt.scatter(x,yO, s=15, c=colors, cmap="Blues") # funziona solo se richiamata singola e non in sequenza
# ## cbar = plt.colorbar()
# ## cbar.set_label("type_stress")
ax.set_xlabel('x label') # Add an x-label to the Axes.
ax.set_ylabel('y label')
ax.legend(bbox_to_anchor=(0., 1.05, 1., .102), loc='lower left',
ncols=2, mode="expand", borderaxespad=0.)# Add a legend.
plt.title("Grafico stress events")
plt.xlabel("Number sequences")
plt.ylabel("stress events")
plt.show(block=False)
plt.pause(100)
plt.close()
#---------------
It should be noted that this model uses a particular library, imported into the Python code with the following instructions:
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, r2_score
Scikit-learn is certainly one of the most important Python libraries for Machine Learning.
This library is completely free, open source, and constantly updated by a vast international community of technicians.
Scikit-learn fits perfectly into this context by providing a wide range of machine learning algorithms for classification, regression, clustering, etc.
Scikit-learn thus represents a powerful and flexible tool for developing machine learning solutions.
Its ease of use and wide range of features make it ideal for a wide range of applications in the field of artificial intelligence, and therefore it seemed appropriate to test some of its capabilities.
The result obtained from this model, in this case, is based on the parameters reported in the terminal:
START LINEAR REGRESSION
*************************
MSE: 102.35064800959468
R-squared: 0.9418871294280144
Intercept (β₀): -147.2268254762597
Coefficient β₁: -0.1065681845316534
Coefficient β₂: -0.23586994510034032
Where MSE indicates the "Mean Squared Error" and R² indicates the coefficient of determination (the regression model score) which, just to give an idea, behaves as follows in MLR:
y_true = [[0.5, 1], [-1, 1], [7, -6]]
y_pred = [[0, 2], [-1, 2], [8, -5]]
r2_score(y_true, y_pred, multioutput='variance_weighted')
0.938...
Considering that in the case under examination, the situation for each input vector would be something like this:
X= [6 2 6 1 3 4 5 2 1 6 4 2 3 3 2 1 6 3 6 2 1 3 5 4 1 3 6 1 1 1 4 2 6 4 5 1 5 6 4 3 5 4 6 2 2 5 6 2 3 4 4 5 1 1 1 2 2 2 2 5 2 5 2 2 4 4 4 3 6 3 3 1 1 3 1 4 1 4 2 4 2 1 3 5 4 3 6 4 3 2 3 1 4 6 2 4 2 6 5 1 6 2 4 5 6 5 6 4 6 5 11 5 6 6 6 6 4 4 1 6 5 4 6 1 5 6 1 4 4 4 1 5 5 6 1 4 5 6 6 5 4 4 4 5 6 6 4 5 6 5 5 6 4 5 4 4 6 4 4 4 4 4 6 4 6 5 4 5 4 6 6 5 4 4 5 6 6 5 6 6 4 5 4 6 6 5 4 5 6 5 6 5 6 5 6 4 6 4 5 5 6 6 6 6 5 4 4 5 4 5 6 4 6 5 6 6 4 6 4 4 4 6 6 6 5 4 4 5 5 4 4 5 4 5 5 5 5 5 4 6 5 4 4 4 5 5 4 5 6 5 5 5 6 5 6 5 5 5 5 6 6 6 6 5 5 5 5 6 6 6 5 5 6 5 6 6 6 5 6 6 5 5 5 6 6 6 5 5 5 6 6 5 6 5 6 5 5 5 5 6 6 6 6 6 6 6 5 6 6 6 6 5 6 5 5 5 5 2 3 6 5 3 2 6 5 2 4 2 3 6 5 3 4 2 5 3 5 6 5 4 2 2 5 3 6 6 4 5 5 3 6 6 5 6 3 4 5 4 5]
stress_test= 124
presumed stress= [113.3332015]
In this case therefore, the graphic output is the following:
where the calculated test vector for stress (y_test) = [37, 112, 42, 124, 134, 132, 117, 62, 134, 124]
while the "raw output vector of the model" is the result of the predictions: [array([56.57256574]), array([128.59989617]), array([29.2146974]), array([124.54476604]), array([134.16260022]), array([125.76638144]), array([117.98344887]), array([72.66654165]), array([120.94963311]), array([113.3332015])]
which has obviously been appropriately processed to calculate the "delta" function, as reported by the instructions:
#----------------
n_k =[]
e=[]
num_test = len(y_test)
for n in range(num_test):
X = X_test[n]
print ("X=",X)
print("stress_test=",y_test[n])
k= model_multi.predict([X])
print ("stress presunto=",k)
n_k.append(k)
delta = ((y_test[n] - k)/y_test[n])*100
e.append(delta)
#--------------------
But there are other models to test using machine learning.
Decision Trees Model
Decision Trees (Dts) are a non-parametric supervised learning method used for classification and regression.
The goal is to create a model that predicts the value of a target variable by learning simple decision rules inferred from the data features.
A tree can be seen as a piecewise constant approximation.
Some advantages of decision trees may be given in reference to the following observations:
This is a relatively simple model to understand and interpret; trees can be visualized.
It requires little data preparation; other techniques often require data normalization, dummy variables creation, and removal of missing values. (Some combinations of trees and algorithms support missing values).
The "cost of using" the tree (i.e., predicting data) is logarithmic in the number of data points used to train the tree.
This model might be able to handle both numerical and categorical data. However, the Scikit-Learn implementation does not support categorical variables for now.
This model is capable of handling multi-output problems.
This is a model that uses explicit logic or can be defined as a "white box." If a particular situation is observable in a model, the explanation of the condition is easily explained by Boolean logic. In contrast, in a black box model (e.g., in an artificial neural network), the results can be more difficult to interpret.
There is the possibility of validating a model using statistical tests. This allows accounting for the reliability of the model.
It is a model that performs well even if its assumptions are somewhat violated by the true model from which the data were generated.
But let's see the code for using this model:
Python code for Decision Tree Model
# analisi vettoriale nota: porta al grafico
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import math
import os
# print("Current working directory: {0}".format(os.getcwd()))
#os.chdir('/home/romeo/2025/2025_python/MWS')
# print("Current working directory: {0}".format(os.getcwd()))
# datset train extaction
train_set = pd.read_csv("Train_seq_num.csv", header = None)
# print("train_set ",train_set.head(3))
# print("len columns ",len(train_set.columns))
# print("last columns", train_set[360]) # stress vector
trv_s = np.array(train_set[360])
#print ("train_vector_stress", train_vector_stress)
vtr_g=np.array(train_set)
#print("len vtrg",len(vtr_g))
vtr= vtr_g[0]
vtr = vtr[:-1]
#------
# print("len_vtr=",len(vtr))
# print("len_stress_vector=",len(trv_s))
#print("primo vettore", vtr)
vtr= vtr_g[len(trv_s)-1]
vtr = vtr[:-1]
#print("ultimo vettore", vtr)
tr_v=[]
for tr in range (len(trv_s)):
vtr= vtr_g[tr]
vtr = vtr[:-1]
tr_v.append(vtr)
#print("len_tr_v", len(tr_v))
#--------------------------
X_train = tr_v
y_train = trv_s
#--------------------------
# datset test extaction
test_set = pd.read_csv("Test_seq_num.csv", header = None)
# print("test_set ",test_set.head())
# print("len columns ",len(test_set.columns))
# print("last columns", test_set[360]) # stress vector
tev_s = np.array(test_set[360])
#print ("train_vector_stress", train_vector_stress)
vte_g=np.array(test_set)
#print("len vte_g",len(vte_g))
vte= vte_g[0]
vte = vte[:-1]
#------
#print("len_vte=",len(vte))
#print("len_stress_v_test=",len(tev_s))
#print("primo vettore", vtr)
vte= vte_g[len(tev_s)-1]
vte = vte[:-1]
#print("ultimo vettore", vtr)
te_v=[]
for te in range (len(tev_s)):
vte= vte_g[te]
vte= vte[:-1]
te_v.append(vte)
#print("len_te_v", len(te_v))
#--------------------------
X_test = te_v
y_test = tev_s
#print ("te_v", te_v)
#print ("te_s", tev_s)
#*********************************#
import sklearn
from sklearn import tree
from sklearn.tree import DecisionTreeClassifier
model_one = sklearn.tree.DecisionTreeClassifier()
model_one.fit(X_train,y_train)
pr1 = model_one.predict(X_test)
print("-------------------------")
#----------------
n_k =[]
e=[]
num_test = len(y_test)
for n in range(num_test):
delta = ((y_test[n] - pr1[n])/y_test[n])*100
e.append(delta)
#------------------
tree.plot_tree(model_one)
# Creazione del grafico
fig, ax = plt.subplots(figsize=(10, 6.10), layout='constrained')
ax.plot(y_test, color="green", marker="s", label='real_test', linestyle="")
ax.plot(pr1, color="red", marker="*",label='prediction',linestyle="")
ax.plot(e, color="gray", marker="*",label='error %',linestyle="-")
ax.set_xlabel('x label') # Add an x-label to the Axes.
ax.set_ylabel('y label')
ax.legend(bbox_to_anchor=(0., 1.05, 1., .102), loc='lower left',
ncols=2, mode="expand", borderaxespad=0.)# Add a legend.
plt.title("Grafico stress events")
plt.xlabel("Number sequences")
plt.ylabel("stress events")
plt.show(block=False)
plt.pause(100)
plt.close()
#---------------------
print ("vettore calcolato di test per stress", y_test)
print ("risultato delle predizioni", pr1)
The code is complete with the data extraction part carried out with Pandas.
Graphic outputs:
As you can see, it is also possible to print the generated tree!
Terminal output:
Calculated test vector for stress [37, 112, 42, 124, 134, 132, 117, 62, 134, 124]
Result of predictions [43, 120, 45, 136, 138, 107, 124, 54, 120, 60]
KNN models
The k-nearest neighbors (KNN) algorithm is a non-parametric supervised learning classifier that uses proximity to make classifications or predictions about the grouping of a single data point.
It is one of the most popular and simple algorithms currently used in machine learning and also has a version that employs it as a regression algorithm.
Although the KNN algorithm can be used for both regression and classification problems, it is usually used as a classification algorithm, based on the assumption that similar points can be found close to each other.
Regression problems use a concept similar to classification, but in this case, the mean of the "k" nearest neighbors is considered to make a prediction about a classification.
The main distinction here is that classification is used for discrete values, while regression is used for continuous ones.
However, before a classification can be made, the distance must be defined.
The Euclidean distance is the most commonly used.
The code for the two KNN models used is as follows:
code for KNN_R and for KNN_C
import sklearn
from sklearn import tree
from sklearn.neighbors import KNeighborsClassifier
from sklearn import neighbors
model_two = sklearn.neighbors.KNeighborsClassifier()
model_two.fit(X_train,y_train)
pr2 = model_two.predict(X_test)
print("-------------------------")
n_neighbors = 1
y_p =[]
for i, weights in enumerate(["uniform", "distance"]):
knn = neighbors.KNeighborsRegressor(n_neighbors, weights=weights)
y_pred = knn.fit(X_train, y_train).predict(X_test)
y_p.append(y_pred)-
knrp=[]
for y_p in range(len(y_pred)):
p = math.floor(y_pred[y_p])
knrp.append(p)
n_k =[]
e=[]
e1=[]
num_test = len(y_test)
for n in range(num_test):
delta = ((y_test[n] - pr2[n])/y_test[n])*100
delta1 = ((y_test[n] - knrp[n])/y_test[n])*100
e.append(delta)
e1.append(delta1)
fig, ax = plt.subplots(figsize=(10, 6.10), layout='constrained')
ax.plot(y_test, color="green", marker="s", label='real_test', linestyle="")
ax.plot(knrp, color="magenta", marker="*", label='KnR_pred', linestyle="")
ax.plot(pr2, color="red", marker="*",label='KnC_pred',linestyle="")
ax.plot(e, color="red", marker="*",label='error_KnC %',linestyle="-")
ax.plot(e1, color="magenta", marker="*",label='error_KnR %',linestyle="-")
ax.set_xlabel('x label') # Add an x-label to the Axes.
ax.set_ylabel('y label')
ax.legend(bbox_to_anchor=(0., 1.05, 1., .102), loc='lower left',
ncols=2, mode="expand", borderaxespad=0.)# Add a legend.
plt.title("Grafico stress events")
plt.xlabel("Number sequences")
plt.ylabel("stress events")
plt.show(block=False)
plt.pause(100)
plt.close()
# #---------------------
print ("vettore calcolato di test per stress", y_test)
print ("y_p", knrp)
print ("pr2", pr2)
Graphic output:
It's interesting to note that the classifier algorithm seems to have slightly better performance than the regressor, but in this case, I encourage the reader to conduct further tests.
Terminal output:
Calculated test vector for stress [37, 112, 42, 124, 134, 132, 117, 62, 134, 124]
y_p [54, 130, 23, 130, 132, 141, 147, 53, 146, 107]
pr2 [54, 114, 16, 128, 132, 130, 119, 54, 121, 134]
Another model to consider could be the logistic regression model, which is also a classification model.
It is used in machine learning to train an algorithm to correctly classify data.
It is a linear model for binary or multi-class classification.
The LLR (LinearLogisticRegression) model
Both linear regression and logistic regression use mathematical models to predict the value of an output variable from one or more input variables.
For linear regression, each independent variable has a direct relationship with the dependent variable and has no relationship with other independent variables.
This relationship is known as a linear relationship.
The dependent variable is typically a value within a range of continuous values.
This is the formula, or linear function, to create a linear regression model:
y= β0 + β1X1 + β2X2+… βnXn+ ε
Here's what each variable means:
• y is the predicted dependent variable
• β0 is the y-intercept when all independent input variables are equal to 0
• β1X1 is the regression coefficient (B1) of the first independent variable (X1), the impact value of the first independent variable on the dependent variable
• βnXn is the regression coefficient (BN) of the last independent variable (XN), when there are multiple input values
• ε is the model error
For logistic regression, on the other hand, the value of the dependent variable comes from a list of finite categories using binary classification.
These are called categorical variables.
An example is the outcome of rolling a six-sided die.
This relationship is known as a logistic relationship.
The formula for logistic regression applies a "logit" transformation, or the natural logarithm of the odds, to the probability of success or failure of a particular categorical variable.
In this case, the formula is as follows:
y = e^(β0 + β1X1 + β2X2+… βnXn+ ε) / (1 + e^(β0 + β1 x 1 + β2 x 2 +… βn x n + ε))
Here is what each variable means:
( y ) gives the probability of success of the categorical variable ( y )
( e(x) ) is Euler's number, the inverse of the natural logarithmic function or the sigmoid function, ( ln(x) )
(β0, β1X1... βnXn) have the same meaning as in linear regression in the previous section.
It is important to emphasize that a statistical model fitting input data to output data does not necessarily imply a causal relationship between the dependent and independent variables. For both logistic regression and linear regression, correlation is not causation.
In any case, the output of linear regression consists of a scale of continuous values (floating-point numbers), while the numerical output of logistic regression, which is the predicted probability, can be used as a classifier by applying a threshold (by default, 0.5).
This is how it is implemented in Scikit-Learn, so it expects a categorical target, making logistic regression a classifier.
The code for the LLR
import sklearn
from sklearn import tree
from sklearn.neighbors import KNeighborsClassifier
model_three = sklearn.linear_model.LogisticRegression()
model_three.fit(X_train,y_train)
pr3 = model_three.predict(X_test)
print("-------------------------")
#----------------
n_k =[]
e=[]
num_test = len(y_test)
for n in range(num_test):
delta = ((y_test[n] - pr3[n])/y_test[n])*100
e.append(delta)
#------------------
# Creazione del grafico
fig, ax = plt.subplots(figsize=(10, 6.10), layout='constrained')
ax.plot(y_test, color="green", marker="s", label='real_test', linestyle="")
ax.plot(pr3, color="red", marker="*",label='prediction',linestyle="")
ax.plot(e, color="gray", marker="*",label='error %',linestyle="-")
ax.set_xlabel('x label') # Add an x-label to the Axes.
ax.set_ylabel('y label')
ax.legend(bbox_to_anchor=(0., 1.05, 1., .102), loc='lower left',
ncols=2, mode="expand", borderaxespad=0.)# Add a legend.
plt.title("Grafico stress events")
plt.xlabel("Number sequences")
plt.ylabel("stress events")
plt.show(block=False)
plt.pause(100)
plt.close()
#---------------------
print ("vettore calcolato di test per stress", y_test)
print ("risultato delle predizioni", pr3)
Graphic Output:
output to terminal:
calculated stress test vector [ 37 112 42 124 134 132 117 62 134 124]
prediction result [ 60 130 38 124 134 134 124 138 127 122]
Conclusions
At this point, I can only invite the reader to explore the world of Machine Learning and Deep Learning, to seek further predictive systems by playing with sequence analysis, also considering the fact that there are multiple fields of application where artificial intelligence can be employed in measurement, monitoring, and control systems.
In this regard, I invite those who wish to delve deeper into these topics, especially if interested in navigating the world of new professions where certain scientific skills are fundamental, to read my book "MMS: Measures Monitoring Control", a text conceived for scientific dissemination with an educational character.
This book covers topics such as plant and animal sensor technology as a starting point, then develops into the field of measurement, monitoring, and control processes, spanning various contexts such as computer science, physics, and medicine. It also addresses topics in cybernetics and artificial intelligence, with particular attention to the fundamental mathematical concepts related to this context.
A book written with the passion of someone who, from a young age, has used the engine of curiosity to navigate the paths that life has offered, in order to try to understand, as much as possible, the play of dynamics that govern events. A book in which some of the most beautiful fruits of the work of brilliant minds have been gathered as universal gifts. A book for all those who, in various capacities, may be attracted to scientific discussions, but which can be particularly recommended as a reference point for students approaching studies in new professions that require a multidisciplinary and multifunctional approach, as implied by the acronym STEM (science, technology, engineering, and mathematics).
In the context of monitoring and control in work environments, for certain tasks such as those of "Stowers" or "Pickers" who must follow sequences dictated by an algorithm, it would be great if repetitions of sequences with a high "Stress" value could be avoided. Similarly, it would be beneficial, at the end of the work shift, to receive a question about the level of stress accumulated, perhaps by clicking on an appropriate "emoticon".
This way, it could give rise to a "Closed-Loop" control system. 😎
Happy reading
Romeo Ceccato
No comments:
Post a Comment