深度学习每周学习总结R3（LSTM-火灾温度推测）

• 🍨 本文为🔗365天深度学习训练营 中的学习记载博客R4中的内容，为了便于本身整理总结起名为R3
  
• 🍖 原作者：K同砚啊 | 接辅导、项目定制
  

  

0. 总结

数据导入及处置惩罚部门：在 PyTorch 中，我们通常先将 NumPy 数组转换为 torch.Tensor，再封装到 TensorDataset 或自界说的 Dataset 里，然后用 DataLoader 按批次加载。
模子构建部门：LSTM
设置超参数：在这之前必要界说丧失函数，学习率（动态学习率），以及根据学习率界说优化器（比方SGD随机梯度降落），用来在训练中更新参数，最小化丧失函数。
界说训练函数：函数的传入的参数有四个，分别是设置好的DataLoader(),界说好的模子，丧失函数，优化器。函数内部初始化丧失精确率为0，接着开始循环，利用DataLoader()获取一个批次的数据，对这个批次的数据带入模子得到推测值，然后利用丧失函数盘算得到丧失值。接下来就是举行反向流传以及利用优化器优化参数，梯度清零放在反向流传之前大概是利用优化器优化之后都是可以的，一样平常是默认放在反向流传之前。
界说测试函数：函数传入的参数相比训练函数少了优化器，只需传入设置好的DataLoader(),界说好的模子，丧失函数。别的除了处置惩罚批次数据时无需再设置梯度清零、返向流传以及优化器优化参数，别的部门均和训练函数保持划一。
训练过程：界说训练次数，有频频就利用整个数据集举行频频训练，初始化四个空list分别存储每次训练及测试的精确率及丧失。利用model.train()开启训练模式，调用训练函数得到精确率及丧失。利用model.eval()将模子设置为评估模式，调用测试函数得到精确率及丧失。接着就是将得到的训练及测试的精确率及丧失存储到相应list中并归并打印出来，得到每一次团体训练后的精确率及丧失。
效果可视化
模子的生存，调取及利用。在PyTorch中，通常利用 torch.save(model.state_dict(), ‘model.pth’) 生存模子的参数，利用 model.load_state_dict(torch.load(‘model.pth’)) 加载参数。
必要改进优化的地方：确保模子和数据的划一性，都存到GPU大概CPU;留意numclasses不要直接用默认的1000，必要根据现实数据集改进；实例化模子也要留意numclasses这个参数；别的留意测试模子必要用（3,224,224）3体现通道数，这和tensorflow界说的次序是不消的（224,224,3），做代码转换时必要留意。
1. LSTM先容

LSTM（Long Short-Term Memory）是一种特殊范例的循环神经网络（RNN），紧张用于处置惩罚和推测基于时间序列的数据。它办理了传统RNN在处置惩罚长时间依靠标题时的范围性，好比“梯度消散”标题。
LSTM的根本构成部门

LSTM通过“影象单元”来决定哪些信息必要记着，哪些必要忘记。它的核心是一个“门控”机制，紧张有三个“门”：

• 忘记门（Forget Gate）：决定上一时间的状态必要忘记多少。
  
• 输入门（Input Gate）：决定当前输入信息中有多少被存储到“影象单元”。
  
• 输出门（Output Gate）：决定“影象单元”中的信息有多少会通报到下一时间的状态。
  

怎样明白与应用LSTM

• 明白LSTM的上风：LSTM能捕获长时间序列中的依靠关系，适当处置惩罚那些长序列的数据，如文本、语音、金融数据等。相比于传统的RNN，LSTM可以大概有用办理“梯度消散”标题，使得模子可以大概学习恒久的依靠关系。
  
• 应用场景：LSTM广泛应用于自然语言处置惩罚（如文本天生、呆板翻译、感情分析）、语音辨认、时间序列推测（如股市推测）等范畴。
  
• 怎样利用LSTM：在初学者阶段，可以从以下几个步调入手：
  
  
  
  • 数据预处置惩罚：将数据转换成得其时间序列建模的格式，好比文本序列或一连的时间戳数据。
    
  • 选择框架：利用像TensorFlow或PyTorch如许的深度学习框架来实现LSTM。你可以从简朴的LSTM网络开始，渐渐增长网络复杂性。
    
  • 调试与优化：调解LSTM的超参数（如潜伏单元数、学习率、批量巨细等）来提升模子性能。
    
  
  

你可以从一个简朴的文本天生模子大概时间序列推测模子开始，渐渐明白LSTM的细节和上风。

1. import torch.nn.functional as F
2. import numpy  as np
3. import pandas as pd
4. import torch
5. from torch import nn
7. import copy
8. import numpy as np
9. import pandas as pd
10. import seaborn as sns
11. import matplotlib.pyplot as plt
12. import warnings
13. warnings.filterwarnings('ignore')
15. from sklearn.model_selection import train_test_split
16. from sklearn.preprocessing import MinMaxScaler
17. from sklearn.metrics import classification_report,confusion_matrix
18. from sklearn.metrics import r2_score
19. from sklearn.metrics import mean_absolute_error , mean_absolute_percentage_error , mean_squared_error

复制代码

2. 数据导入

1. data = pd.read_csv("./data/woodpine2.csv")
2. data

复制代码

  TimeTem1CO 1Soot 100.00025.00.0000000.00000010.22825.00.0000000.00000020.45625.00.0000000.00000030.68525.00.0000000.00000040.91325.00.0000000.000000...............5943366.000295.00.0000770.0004965944366.000294.00.0000770.0004945945367.000292.00.0000770.0004915946367.000291.00.0000760.0004895947367.000290.00.0000760.000487 5948 rows × 4 columns

1. plt.rcParams['savefig.dpi'] = 500 #图片像素
2. plt.rcParams['figure.dpi']  = 500 #分辨率
4. fig, ax =plt.subplots(1,3,constrained_layout=True, figsize=(14, 3))
6. sns.lineplot(data=data["Tem1"], ax=ax[0])
7. sns.lineplot(data=data["CO 1"], ax=ax[1])
8. sns.lineplot(data=data["Soot 1"], ax=ax[2])
9. plt.show()

复制代码

​

​

1. dataFrame = data.iloc[:,1:]
2. dataFrame

复制代码

  Tem1CO 1Soot 1025.00.0000000.000000125.00.0000000.000000225.00.0000000.000000325.00.0000000.000000425.00.0000000.000000............5943295.00.0000770.0004965944294.00.0000770.0004945945292.00.0000770.0004915946291.00.0000760.0004895947290.00.0000760.000487 5948 rows × 3 columns
3. 数据预处置惩罚

1. dataFrame = data.iloc[:,1:].copy()
2. sc  = MinMaxScaler(feature_range=(0, 1)) #将数据归一化，范围是0到1
4. for i in ['CO 1', 'Soot 1', 'Tem1']:
5.     dataFrame[i] = sc.fit_transform(dataFrame[i].values.reshape(-1, 1))
7. dataFrame.shape

复制代码

1. (5948, 3)

复制代码

1. # 设置X、y
2. width_X = 8
3. width_y = 1
5. ##取前8个时间段的Tem1、CO 1、Soot 1为X，第9个时间段的Tem1为y。
6. X = []
7. y = []
9. in_start = 0
11. for _, _ in data.iterrows():
12.     in_end  = in_start + width_X
13.     out_end = in_end   + width_y
14.    
15.     if out_end < len(dataFrame):
16.         X_ = np.array(dataFrame.iloc[in_start:in_end , ])
17.         y_ = np.array(dataFrame.iloc[in_end  :out_end, 0])
19.         X.append(X_)
20.         y.append(y_)
21.    
22.     in_start += 1
24. X = np.array(X)
25. y = np.array(y).reshape(-1,1,1)
27. X.shape, y.shape

复制代码

1. ((5939, 8, 3), (5939, 1, 1))

复制代码

1. # 检查数据集中是否有空值
2. print(np.any(np.isnan(X)))
3. print(np.any(np.isnan(y)))

复制代码

1. False
2. False

复制代码

4. 分别数据集

1. X_train = torch.tensor(np.array(X[:5000]), dtype=torch.float32)
2. y_train = torch.tensor(np.array(y[:5000]), dtype=torch.float32)
4. X_test  = torch.tensor(np.array(X[5000:]), dtype=torch.float32)
5. y_test  = torch.tensor(np.array(y[5000:]), dtype=torch.float32)
6. X_train.shape, y_train.shape

复制代码

1. (torch.Size([5000, 8, 3]), torch.Size([5000, 1, 1]))

复制代码

1. from torch.utils.data import TensorDataset, DataLoader
3. train_dl = DataLoader(TensorDataset(X_train, y_train),
4.                       batch_size=64,
5.                       shuffle=False)
7. test_dl  = DataLoader(TensorDataset(X_test, y_test),
8.                       batch_size=64,
9.                       shuffle=False)

复制代码

5. 模子构建

1. class model_lstm(nn.Module):
2.     def __init__(self):
3.         super(model_lstm, self).__init__()
4.         self.lstm0 = nn.LSTM(input_size=3 ,hidden_size=320,
5.                              num_layers=1, batch_first=True)
6.         
7.         self.lstm1 = nn.LSTM(input_size=320 ,hidden_size=320,
8.                              num_layers=1, batch_first=True)
9.         self.fc0   = nn.Linear(320, 1)
11.     def forward(self, x):
13.         out, hidden1 = self.lstm0(x)
14.         out, _ = self.lstm1(out, hidden1)
15.         out    = self.fc0(out)
16.         return out[:, -1:, :]   #取2个预测值,否则经过lstm会得到8*2个预测

复制代码

6. 初始化模子及优化器

1. device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
3. model = model_lstm()
4. print(model)
6. epochs = 50
7. loss_fn    = nn.MSELoss() # 创建损失函数
8. learn_rate = 1e-1   # 学习率
9. opt        = torch.optim.SGD(model.parameters(),lr=learn_rate,weight_decay=1e-4)
10. lr_scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(opt,epochs, last_epoch=-1) # 选定调整方法

复制代码

1. model_lstm(
2.   (lstm0): LSTM(3, 320, batch_first=True)
3.   (lstm1): LSTM(320, 320, batch_first=True)
4.   (fc0): Linear(in_features=320, out_features=1, bias=True)
5. )

复制代码

1. # 观察模型的输出数据集格式是什么
2. model(torch.rand(30,8,3)).shape

复制代码

1. torch.Size([30, 1, 1])

复制代码

7. 界说训练函数

1. def train(train_dl, model, loss_fn, opt, lr_scheduler=None):
2.     size        = len(train_dl.dataset)  
3.     num_batches = len(train_dl)   
4.     train_loss  = 0  # 初始化训练损失和正确率
5.    
6.     for x, y in train_dl:  
7.         x, y = x.to(device), y.to(device)
8.         
9.         # 计算预测误差
10.         pred = model(x)          # 网络输出
11.         loss = loss_fn(pred, y)  # 计算网络输出和真实值之间的差距
12.         
13.         # 反向传播
14.         opt.zero_grad()  # grad属性归零
15.         loss.backward()  # 反向传播
16.         opt.step()       # 每一步自动更新
17.         
18.         # 记录loss
19.         train_loss += loss.item()
20.         
21.     if lr_scheduler is not None:
22.         lr_scheduler.step()
23.         print("learning rate = {:.5f}".format(opt.param_groups[0]['lr']), end="  ")
24.     train_loss /= num_batches
25.     return train_loss

复制代码

8. 界说测试函数

1. def test (dataloader, model, loss_fn):
2.     size        = len(dataloader.dataset)  # 测试集的大小
3.     num_batches = len(dataloader)          # 批次数目
4.     test_loss   = 0
5.    
6.     # 当不进行训练时，停止梯度更新，节省计算内存消耗
7.     with torch.no_grad():
8.         for x, y in dataloader:
9.             
10.             x, y = x.to(device), y.to(device)
11.             
12.             # 计算loss
13.             y_pred = model(x)
14.             loss        = loss_fn(y_pred, y)
15.             test_loss += loss.item()
16.         
17.     test_loss /= num_batches
18.     return test_loss

复制代码

9. 训练过程

1. train_loss = []
2. test_loss  = []
4. for epoch in range(epochs):
5.     model.train()
6.     epoch_train_loss = train(train_dl, model, loss_fn, opt, lr_scheduler)
8.     model.eval()
9.     epoch_test_loss = test(test_dl, model, loss_fn)
11.     train_loss.append(epoch_train_loss)
12.     test_loss.append(epoch_test_loss)
13.    
14.     template = ('Epoch:{:2d}, Train_loss:{:.5f}, Test_loss:{:.5f}')
15.     print(template.format(epoch+1, epoch_train_loss,  epoch_test_loss))
16.    
17. print("="*20, 'Done', "="*20)

复制代码

1. learning rate = 0.09990  Epoch: 1, Train_loss:0.00133, Test_loss:0.01258
2. learning rate = 0.09961  Epoch: 2, Train_loss:0.01467, Test_loss:0.01221
3. learning rate = 0.09911  Epoch: 3, Train_loss:0.01437, Test_loss:0.01183
4. learning rate = 0.09843  Epoch: 4, Train_loss:0.01403, Test_loss:0.01142
5. learning rate = 0.09755  Epoch: 5, Train_loss:0.01363, Test_loss:0.01097
6. learning rate = 0.09649  Epoch: 6, Train_loss:0.01317, Test_loss:0.01046
7. learning rate = 0.09524  Epoch: 7, Train_loss:0.01262, Test_loss:0.00988
8. learning rate = 0.09382  Epoch: 8, Train_loss:0.01197, Test_loss:0.00924
9. learning rate = 0.09222  Epoch: 9, Train_loss:0.01120, Test_loss:0.00852
10. learning rate = 0.09045  Epoch:10, Train_loss:0.01032, Test_loss:0.00774
11. learning rate = 0.08853  Epoch:11, Train_loss:0.00933, Test_loss:0.00690
12. learning rate = 0.08645  Epoch:12, Train_loss:0.00826, Test_loss:0.00605
13. learning rate = 0.08423  Epoch:13, Train_loss:0.00712, Test_loss:0.00519
14. learning rate = 0.08187  Epoch:14, Train_loss:0.00598, Test_loss:0.00438
15. learning rate = 0.07939  Epoch:15, Train_loss:0.00488, Test_loss:0.00362
16. learning rate = 0.07679  Epoch:16, Train_loss:0.00387, Test_loss:0.00296
17. learning rate = 0.07409  Epoch:17, Train_loss:0.00298, Test_loss:0.00240
18. learning rate = 0.07129  Epoch:18, Train_loss:0.00224, Test_loss:0.00194
19. learning rate = 0.06841  Epoch:19, Train_loss:0.00165, Test_loss:0.00158
20. learning rate = 0.06545  Epoch:20, Train_loss:0.00120, Test_loss:0.00130
21. learning rate = 0.06243  Epoch:21, Train_loss:0.00087, Test_loss:0.00110
22. learning rate = 0.05937  Epoch:22, Train_loss:0.00063, Test_loss:0.00095
23. learning rate = 0.05627  Epoch:23, Train_loss:0.00047, Test_loss:0.00084
24. learning rate = 0.05314  Epoch:24, Train_loss:0.00035, Test_loss:0.00076
25. learning rate = 0.05000  Epoch:25, Train_loss:0.00027, Test_loss:0.00070
26. learning rate = 0.04686  Epoch:26, Train_loss:0.00022, Test_loss:0.00066
27. learning rate = 0.04373  Epoch:27, Train_loss:0.00018, Test_loss:0.00063
28. learning rate = 0.04063  Epoch:28, Train_loss:0.00016, Test_loss:0.00060
29. learning rate = 0.03757  Epoch:29, Train_loss:0.00014, Test_loss:0.00058
30. learning rate = 0.03455  Epoch:30, Train_loss:0.00013, Test_loss:0.00057
31. learning rate = 0.03159  Epoch:31, Train_loss:0.00012, Test_loss:0.00056
32. learning rate = 0.02871  Epoch:32, Train_loss:0.00012, Test_loss:0.00055
33. learning rate = 0.02591  Epoch:33, Train_loss:0.00011, Test_loss:0.00055
34. learning rate = 0.02321  Epoch:34, Train_loss:0.00011, Test_loss:0.00054
35. learning rate = 0.02061  Epoch:35, Train_loss:0.00011, Test_loss:0.00054
36. learning rate = 0.01813  Epoch:36, Train_loss:0.00011, Test_loss:0.00054
37. learning rate = 0.01577  Epoch:37, Train_loss:0.00012, Test_loss:0.00054
38. learning rate = 0.01355  Epoch:38, Train_loss:0.00012, Test_loss:0.00055
39. learning rate = 0.01147  Epoch:39, Train_loss:0.00012, Test_loss:0.00056
40. learning rate = 0.00955  Epoch:40, Train_loss:0.00013, Test_loss:0.00056
41. learning rate = 0.00778  Epoch:41, Train_loss:0.00013, Test_loss:0.00057
42. learning rate = 0.00618  Epoch:42, Train_loss:0.00013, Test_loss:0.00058
43. learning rate = 0.00476  Epoch:43, Train_loss:0.00013, Test_loss:0.00058
44. learning rate = 0.00351  Epoch:44, Train_loss:0.00014, Test_loss:0.00059
45. learning rate = 0.00245  Epoch:45, Train_loss:0.00014, Test_loss:0.00059
46. learning rate = 0.00157  Epoch:46, Train_loss:0.00014, Test_loss:0.00059
47. learning rate = 0.00089  Epoch:47, Train_loss:0.00014, Test_loss:0.00059
48. learning rate = 0.00039  Epoch:48, Train_loss:0.00014, Test_loss:0.00059
49. learning rate = 0.00010  Epoch:49, Train_loss:0.00014, Test_loss:0.00059
50. learning rate = 0.00000  Epoch:50, Train_loss:0.00014, Test_loss:0.00059
51. ==================== Done ====================

复制代码

10. 模子评估

1. # loss 图
2. plt.figure(figsize=(5, 3),dpi=120)
4. plt.plot(train_loss    , label='LSTM Training Loss')
5. plt.plot(test_loss, label='LSTM Validation Loss')
7. plt.title('Training and Validation Loss')
8. plt.legend()
9. plt.show()

复制代码

​

​
11. 调用模子举行推测

1. predicted_y_lstm = sc.inverse_transform(model(X_test).detach().numpy().reshape(-1,1))                    # 测试集输入模型进行预测
2. y_test_1         = sc.inverse_transform(y_test.reshape(-1,1))
3. y_test_one       = [i[0] for i in y_test_1]
4. predicted_y_lstm_one = [i[0] for i in predicted_y_lstm]
6. plt.figure(figsize=(5, 3),dpi=120)
7. # 画出真实数据和预测数据的对比曲线
8. plt.plot(y_test_one[:2000], color='red', label='real_temp')
9. plt.plot(predicted_y_lstm_one[:2000], color='blue', label='prediction')
11. plt.title('Title')
12. plt.xlabel('X')
13. plt.ylabel('Y')
14. plt.legend()
15. plt.show()

复制代码

​

​
12. R2评估

1. from sklearn import metrics
2. """
3. RMSE ：均方根误差  ----->  对均方误差开方
4. R2   ：决定系数，可以简单理解为反映模型拟合优度的重要的统计量
5. """
6. RMSE_lstm  = metrics.mean_squared_error(predicted_y_lstm_one, y_test_1)**0.5
7. R2_lstm    = metrics.r2_score(predicted_y_lstm_one, y_test_1)
9. print('均方根误差: %.5f' % RMSE_lstm)
10. print('R2: %.5f' % R2_lstm)

复制代码

1. 均方根误差: 6.89830
2. R2: 0.83397

复制代码

[code][/code]