Tutorial on Rainfall-Runoff Modeling with RNNs#
This hands-on tutorial accompanies the Recurrent Neural Networks notes. Recurrent neural networks were developed in the context of natural language processing and work well for sequential data; for environmental data sets, this means they can work well for predicting time series.
In this tutorial, we will train an RNN, LSTM, and GRU model to predict discharge from streams using the CAMELS data set (Newman et al. 2014). The CAMELS data set provides 35 years of daily meteorological forcings and discharge observations from 671 basins across the contiguous United States. As input to the model, we will use meteorological data (total daily precipitation, daily min/max temperature, average solar radiation, and vapor pressure), and we will predict the total discharge from the basin. This is the sequence-to-vector setup from the notes: a full year of past meteorology in, a single day of discharge out.
This is based off of several recent papers that looked at the potential for LSTM’s to be used for rainfall-runoff modeling.
This notebook follows a similar example here by Frederik Kratzert, implemented in PyTorch, which is based off the following sources:
[1] Kratzert, F., Klotz, D., Brenner, C., Schulz, K., and Herrnegger, M.: Rainfall–runoff modelling using Long Short-Term Memory (LSTM) networks, Hydrol. Earth Syst. Sci., 22, 6005-6022, https://doi.org/10.5194/hess-22-6005-2018, 2018a.
[2] Kratzert F., Klotz D., Herrnegger M., Hochreiter S.: A glimpse into the Unobserved: Runoff simulation for ungauged catchments with LSTMs, Workshop on Modeling and Decision-Making in the Spatiotemporal Domain, 32nd Conference on Neural Information Processing Systems (NeuRIPS 2018), Montréal, Canada. https://openreview.net/forum?id=Bylhm72oKX, 2018b.
[3] A. Newman; K. Sampson; M. P. Clark; A. Bock; R. J. Viger; D. Blodgett, 2014. A large-sample watershed-scale hydrometeorological dataset for the contiguous USA. Boulder, CO: UCAR/NCAR. https://dx.doi.org/10.5065/D6MW2F4D.
Note
This notebook reads the CAMELS data from a requester-pays Google Cloud Storage bucket (via gcsfs) and trains with TensorFlow, so it is meant to be run on a cloud platform such as the LEAP JupyterHub (choose the TensorFlow server image) rather than on a laptop. Training is much faster on a GPU. The outputs shown here are from one real run.
# Imports
import os
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "3" # hide TF's cuFFT/cuDNN/cuBLAS startup logs
from pathlib import Path
from typing import Tuple, List
import gcsfs
import matplotlib.pyplot as plt
from numba import njit
import numpy as np
import pandas as pd
# Globals
FILE_SYSTEM = gcsfs.core.GCSFileSystem(requester_pays=True)
CAMELS_ROOT = Path('pangeo-ncar-camels/basin_dataset_public_v1p2')
import warnings
warnings.filterwarnings('ignore')
This is a helper function that will load in the meteorological data for any specific basin from the CAMELS data set. From the header of the forcing file, we can also extract the catchment area, to normalize the discharge (to mm/day)
def load_forcing(basin: str) -> Tuple[pd.DataFrame, int]:
"""Load the meteorological forcing data of a specific basin.
:param basin: 8-digit code of basin as string.
:return: pd.DataFrame containing the meteorological forcing data and the
area of the basin as integer.
"""
# root directory of meteorological forcings
forcing_path = CAMELS_ROOT / 'basin_mean_forcing' / 'daymet'
# get path of forcing file
files = list(FILE_SYSTEM.glob(f"{str(forcing_path)}/**/{basin}_*.txt"))
if len(files) == 0:
raise RuntimeError(f'No forcing file file found for Basin {basin}')
else:
file_path = files[0]
# read-in data and convert date to datetime index
with FILE_SYSTEM.open(file_path) as fp:
df = pd.read_csv(fp, sep=r'\s+', header=3)
dates = (df.Year.map(str) + "/" + df.Mnth.map(str) + "/"
+ df.Day.map(str))
df.index = pd.to_datetime(dates, format="%Y/%m/%d")
# load area from header
with FILE_SYSTEM.open(file_path) as fp:
content = fp.readlines()
area = int(content[2])
return df, area
This is a helper function that loads in the discharge time series for a specific streamflow basin.
def load_discharge(basin: str, area: int) -> pd.Series:
"""Load the discharge time series for a specific basin.
:param basin: 8-digit code of basin as string.
:param area: int, area of the catchment in square meters
:return: A pd.Series containng the catchment normalized discharge.
"""
# root directory of the streamflow data
discharge_path = CAMELS_ROOT / 'usgs_streamflow'
# get path of streamflow file file
files = list(FILE_SYSTEM.glob(f"{str(discharge_path)}/**/{basin}_*.txt"))
if len(files) == 0:
raise RuntimeError(f'No discharge file found for Basin {basin}')
else:
file_path = files[0]
# read-in data and convert date to datetime index
col_names = ['basin', 'Year', 'Mnth', 'Day', 'QObs', 'flag']
with FILE_SYSTEM.open(file_path) as fp:
df = pd.read_csv(fp, sep=r'\s+', header=None, names=col_names)
dates = (df.Year.map(str) + "/" + df.Mnth.map(str) + "/"
+ df.Day.map(str))
df.index = pd.to_datetime(dates, format="%Y/%m/%d")
# normalize discharge from cubic feet per second to mm per day
df.QObs = 28316846.592 * df.QObs * 86400 / (area * 10 ** 6)
return df.QObs
We’ll load in the data for a single basin and visualize the time series.
basin = '01022500'
df, area = load_forcing(basin)
df['QObs(mm/d)'] = load_discharge(basin, area)
df.head()
| Year | Mnth | Day | Hr | dayl(s) | prcp(mm/day) | srad(W/m2) | swe(mm) | tmax(C) | tmin(C) | vp(Pa) | QObs(mm/d) | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1980-01-01 | 1980 | 1 | 1 | 12 | 31185.97 | 0.0 | 178.46 | 0.0 | -0.72 | -10.59 | 271.45 | 1.644439 |
| 1980-01-02 | 1980 | 1 | 2 | 12 | 31302.05 | 0.0 | 186.36 | 0.0 | 0.74 | -9.82 | 291.32 | 1.457098 |
| 1980-01-03 | 1980 | 1 | 3 | 12 | 31370.88 | 0.0 | 198.93 | 0.0 | -0.65 | -12.28 | 238.75 | 1.290572 |
| 1980-01-04 | 1980 | 1 | 4 | 12 | 31449.59 | 0.0 | 182.99 | 0.0 | -5.35 | -15.20 | 191.65 | 1.144863 |
| 1980-01-05 | 1980 | 1 | 5 | 12 | 31449.59 | 0.0 | 155.25 | 0.0 | -7.15 | -14.94 | 191.74 | 1.040784 |
df.columns
Index(['Year', 'Mnth', 'Day', 'Hr', 'dayl(s)', 'prcp(mm/day)', 'srad(W/m2)',
'swe(mm)', 'tmax(C)', 'tmin(C)', 'vp(Pa)', 'QObs(mm/d)'],
dtype='object')
df.info()
<class 'pandas.core.frame.DataFrame'>
DatetimeIndex: 12784 entries, 1980-01-01 to 2014-12-31
Data columns (total 12 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 Year 12784 non-null int64
1 Mnth 12784 non-null int64
2 Day 12784 non-null int64
3 Hr 12784 non-null int64
4 dayl(s) 12784 non-null float64
5 prcp(mm/day) 12784 non-null float64
6 srad(W/m2) 12784 non-null float64
7 swe(mm) 12784 non-null float64
8 tmax(C) 12784 non-null float64
9 tmin(C) 12784 non-null float64
10 vp(Pa) 12784 non-null float64
11 QObs(mm/d) 12784 non-null float64
dtypes: float64(8), int64(4)
memory usage: 1.3 MB
import matplotlib.pyplot as plt
df["QObs(mm/d)"].plot(grid=True,marker=".",figsize = (8,3.5))
plt.ylabel("Discharge (mm/day)")
plt.xlabel("Year")
plt.show()
df["1995":"2005"]["QObs(mm/d)"].plot(grid=True,marker=".",figsize = (8,3.5))
plt.ylabel("Discharge (mm/day)")
plt.xlabel("Year")
plt.show()
Preprocess data sets for machine learning#
We want to predict the stream discharge rate for the future, given the past meteorological data sets. We will first preprocess the data using the StandardScaler from scikit-learn.
targets = df[["QObs(mm/d)"]].copy()
features = df.drop(columns = ['Year', 'Mnth', 'Day', 'Hr', 'dayl(s)','swe(mm)','QObs(mm/d)']).copy()
features.hist()
array([[<Axes: title={'center': 'prcp(mm/day)'}>,
<Axes: title={'center': 'srad(W/m2)'}>],
[<Axes: title={'center': 'tmax(C)'}>,
<Axes: title={'center': 'tmin(C)'}>],
[<Axes: title={'center': 'vp(Pa)'}>, <Axes: >]], dtype=object)
targets.hist()
array([[<Axes: title={'center': 'QObs(mm/d)'}>]], dtype=object)
from sklearn.preprocessing import StandardScaler, MinMaxScaler
target_scaler = StandardScaler()
feature_scaler = StandardScaler()
scaled_targets = target_scaler.fit_transform(targets)
scaled_features = feature_scaler.fit_transform(features)
plt.hist(targets)
plt.hist(scaled_targets)
(array([9.200e+01, 9.043e+03, 2.685e+03, 6.310e+02, 2.000e+02, 7.300e+01,
3.700e+01, 1.100e+01, 5.000e+00, 7.000e+00]),
array([-2.48310772, -1.19094774, 0.10121224, 1.39337221, 2.68553219,
3.97769217, 5.26985214, 6.56201212, 7.85417209, 9.14633207,
10.43849205]),
<BarContainer object of 10 artists>)
fig, axs = plt.subplots(2, 3,figsize=(8,6))
ax = axs.ravel()
for i in range(0,5):
ax[i].hist(scaled_features[:,i])
ax[i].set_ylabel(features.columns[i])
plt.tight_layout()
Reshaping data for RNN training#
We next need to reshape data so that it is in the correct format to train an RNN, LSTM, or GRU model. Recurrent neural networks expect sequential input of the shape (sequence length, number of features). We want to train a model to predict a single day of discharge from n days of previous meteorological observations. For example, if n = 365, then a single training sample should be of the shape (365, number of features). Here we use 5 input features, so the shape would be (365, 5).
However, the time series data is currently stored in a matrix, where the number of rows corresponds to the total number of days in the training data set, and the number of columns is the number of features. We need to slide over this matrix and cut out small samples to act as training samples for the RNN models that we are going to train. Keras and Tensorflow have a couple of different utility functions that can help us with this task. Keras ships tf.keras.utils.timeseries_dataset_from_array for exactly this, but it fails on the TensorFlow build in the LEAP image, so we use a short NumPy helper instead — it takes the same arguments and returns the same (inputs, targets) batches.
import tensorflow as tf
from tensorflow.keras.models import Model, load_model
def windows(data, targets, sequence_length, batch_size, shuffle=False, seed=None):
"""Slide a window of `sequence_length` over `data`, pairing each with a target."""
data, targets = np.asarray(data), np.asarray(targets)
X = np.lib.stride_tricks.sliding_window_view(data, sequence_length, axis=0)
X = np.moveaxis(X, -1, 1)[:len(targets)] # (n_windows, sequence_length, n_features)
ds = tf.data.Dataset.from_tensor_slices((X, targets))
if shuffle:
ds = ds.shuffle(len(targets), seed=seed)
return ds.batch(batch_size)
my_series = [0, 1, 2, 3, 4, 5]
my_dataset = windows(
my_series,
my_series[3:], # the targets are 3 steps into the future
sequence_length=3,
batch_size=2
)
list(my_dataset)
[(<tf.Tensor: shape=(2, 3), dtype=int64, numpy=
array([[0, 1, 2],
[1, 2, 3]])>,
<tf.Tensor: shape=(2,), dtype=int64, numpy=array([3, 4])>),
(<tf.Tensor: shape=(1, 3), dtype=int64, numpy=array([[2, 3, 4]])>,
<tf.Tensor: shape=(1,), dtype=int64, numpy=array([5])>)]
my_dataset
<_BatchDataset element_spec=(TensorSpec(shape=(None, 3), dtype=tf.int64, name=None), TensorSpec(shape=(None,), dtype=tf.int64, name=None))>
for window_dataset in tf.data.Dataset.range(6).window(4, shift=1):
for element in window_dataset:
print(f"{element}", end=" ")
print()
0 1 2 3
1 2 3 4
2 3 4 5
3 4 5
4 5
5
dataset = tf.data.Dataset.range(6).window(4, shift=1, drop_remainder=True)
dataset = dataset.flat_map(lambda window_dataset: window_dataset.batch(4))
for window_tensor in dataset:
print(f"{window_tensor}")
[0 1 2 3]
[1 2 3 4]
[2 3 4 5]
def to_windows(dataset, length):
dataset = dataset.window(length, shift=1, drop_remainder=True)
return dataset.flat_map(lambda window_ds: window_ds.batch(length))
Split data into training, validation, and test data#
In addition to setting up the sequence data sets for training the model, we need to designate part of our time series for training, part for validation, and part for testing. We will use 1980 - 1995 for training, 1995 - 2000 for validation, and 2000 to 2010 as our independent test data set.
trainmask = (df.index >="1980-10-01") & (df.index <="1995-09-30")
valmask = (df.index >="1995-10-01") & (df.index <="2000-09-30")
testmask = (df.index >="2000-10-01") & (df.index <="2010-09-30")
trainidx = np.where(trainmask)[0]
validx = np.where(valmask)[0]
testidx = np.where(testmask)[0]
plt.plot(scaled_targets,color="k")
plt.plot(trainidx,scaled_targets[trainidx],color="g",label="train")
plt.plot(validx,scaled_targets[validx],color="r",label="val")
plt.plot(testidx,scaled_targets[testidx],color="b",label="test")
plt.legend()
<matplotlib.legend.Legend at 0x7e0d95555940>
We will use an entire year of meteorological data as input to predict the next time step.
sequence_length = 365 # Length of the meteorological record provided to the network
tf.random.set_seed(42) # ensures reproducibility
train_ds = windows(
scaled_features[trainidx],
targets=scaled_targets[trainidx][sequence_length - 1:],
sequence_length=sequence_length,
batch_size=256,
shuffle=True,
seed=42
)
valid_ds = windows(
scaled_features[validx],
targets=scaled_targets[validx][sequence_length - 1:],
sequence_length=sequence_length,
batch_size=2048
)
test_ds = windows(
scaled_features[testidx],
targets=scaled_targets[testidx][sequence_length - 1:],
sequence_length=sequence_length,
batch_size=len(testidx)
)
for x, y in train_ds.take(1):
print("Input shape:", x.shape)
print("Target shape:", y.shape)
Input shape: (256, 365, 5)
Target shape: (256, 1)
Train a Simple RNN#
We’ll first try training an RNN model.
import os
cwd = os.getcwd()
model_path = os.path.join(cwd,'saved_model')
# set some hyperparameters
n_hidden = 10
patience = 20
epochs = 100
learning_rate = 1e-3
This code creates the RNN model using tf.keras.Sequential.
tf.random.set_seed(42) # ensures reproducibility
rnn_model = tf.keras.Sequential([
tf.keras.layers.SimpleRNN(n_hidden, input_shape=[None, 5]),
tf.keras.layers.Dense(1)
])
rnn_model.summary()
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩ │ simple_rnn (SimpleRNN) │ (None, 10) │ 160 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dense (Dense) │ (None, 1) │ 11 │ └─────────────────────────────────┴────────────────────────┴───────────────┘
Total params: 171 (684.00 B)
Trainable params: 171 (684.00 B)
Non-trainable params: 0 (0.00 B)
We’ll include a custom metric, the Nash Sutcliffe Efficiency, which is a widely used metric in hydrology for assessing how well a model predicts the observed data.
class NashSutcliffeEfficiency(tf.keras.metrics.Metric):
def __init__(self, name='nse', scaler=None, **kwargs):
super().__init__(name=name, **kwargs)
self.sse = self.add_weight(name='sse', initializer='zeros')
self.sst = self.add_weight(name='sst', initializer='zeros')
self.scaler = scaler
def update_state(self, y_true, y_pred, sample_weight=None):
if self.scaler is not None:
u = self.scaler.mean_
s = self.scaler.var_
y_true = y_true*s+u
y_pred = y_pred*s+u
y_true = tf.cast(y_true, tf.float32)
y_pred = tf.cast(y_pred, tf.float32)
sse = tf.reduce_sum(tf.square(y_true - y_pred))
sst = tf.reduce_sum(tf.square(y_true - tf.reduce_mean(y_true)))
self.sse.assign_add(sse)
self.sst.assign_add(sst)
def result(self):
return 1.0 - self.sse / self.sst
def reset_states(self):
self.sse.assign(0.0)
self.sst.assign(0.0)
early_stopping_cb = tf.keras.callbacks.EarlyStopping(
monitor="val_loss", patience=patience, restore_best_weights=True)
opt = tf.keras.optimizers.Adam(learning_rate=learning_rate)
rnn_model.compile(loss='mse', optimizer=opt, metrics=[NashSutcliffeEfficiency(scaler=target_scaler)])
history = rnn_model.fit(train_ds, validation_data=valid_ds, epochs=epochs,callbacks=[early_stopping_cb])
rnn_model.save(os.path.join(model_path,'RNN_timeseries_model.keras'))
Epoch 1/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 5s 121ms/step - loss: 1.1340 - nse: -0.3406 - val_loss: 0.9037 - val_nse: -0.0985
Epoch 2/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 44ms/step - loss: 0.9300 - nse: -0.0687 - val_loss: 0.7581 - val_nse: 0.0785
Epoch 3/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 44ms/step - loss: 0.8025 - nse: 0.0706 - val_loss: 0.6561 - val_nse: 0.2025
Epoch 4/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.6915 - nse: 0.2046 - val_loss: 0.5800 - val_nse: 0.2950
Epoch 5/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.5788 - nse: 0.2940 - val_loss: 0.5227 - val_nse: 0.3646
Epoch 6/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.5102 - nse: 0.3470 - val_loss: 0.4942 - val_nse: 0.3993
Epoch 7/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 44ms/step - loss: 0.5330 - nse: 0.3967 - val_loss: 0.4681 - val_nse: 0.4310
Epoch 8/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 44ms/step - loss: 0.4456 - nse: 0.4488 - val_loss: 0.4472 - val_nse: 0.4564
Epoch 9/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.4296 - nse: 0.4547 - val_loss: 0.4292 - val_nse: 0.4782
Epoch 10/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 44ms/step - loss: 0.4210 - nse: 0.4834 - val_loss: 0.4245 - val_nse: 0.4840
Epoch 11/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.4246 - nse: 0.4882 - val_loss: 0.4138 - val_nse: 0.4970
Epoch 12/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.3975 - nse: 0.5197 - val_loss: 0.4092 - val_nse: 0.5026
Epoch 13/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 47ms/step - loss: 0.4370 - nse: 0.5017 - val_loss: 0.4005 - val_nse: 0.5132
Epoch 14/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.3700 - nse: 0.5340 - val_loss: 0.3970 - val_nse: 0.5174
Epoch 15/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 44ms/step - loss: 0.3834 - nse: 0.5324 - val_loss: 0.3926 - val_nse: 0.5227
Epoch 16/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.4201 - nse: 0.5195 - val_loss: 0.3906 - val_nse: 0.5252
Epoch 17/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.4452 - nse: 0.5175 - val_loss: 0.3846 - val_nse: 0.5325
Epoch 18/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 44ms/step - loss: 0.3994 - nse: 0.5498 - val_loss: 0.3810 - val_nse: 0.5369
Epoch 19/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.4089 - nse: 0.5381 - val_loss: 0.3695 - val_nse: 0.5508
Epoch 20/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.3987 - nse: 0.5420 - val_loss: 0.3725 - val_nse: 0.5472
Epoch 21/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.3850 - nse: 0.5504 - val_loss: 0.3674 - val_nse: 0.5534
Epoch 22/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.3713 - nse: 0.5713 - val_loss: 0.3666 - val_nse: 0.5544
Epoch 23/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 45ms/step - loss: 0.4318 - nse: 0.5539 - val_loss: 0.3598 - val_nse: 0.5627
Epoch 24/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 44ms/step - loss: 0.3660 - nse: 0.5696 - val_loss: 0.3533 - val_nse: 0.5705
Epoch 25/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 44ms/step - loss: 0.3662 - nse: 0.5791 - val_loss: 0.3569 - val_nse: 0.5662
Epoch 26/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 44ms/step - loss: 0.3769 - nse: 0.5705 - val_loss: 0.3525 - val_nse: 0.5715
Epoch 27/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.3968 - nse: 0.5690 - val_loss: 0.3514 - val_nse: 0.5728
Epoch 28/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.3225 - nse: 0.6078 - val_loss: 0.3478 - val_nse: 0.5772
Epoch 29/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.3818 - nse: 0.5800 - val_loss: 0.3432 - val_nse: 0.5829
Epoch 30/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.3390 - nse: 0.5928 - val_loss: 0.3451 - val_nse: 0.5806
Epoch 31/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.3343 - nse: 0.6066 - val_loss: 0.3439 - val_nse: 0.5819
Epoch 32/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.3504 - nse: 0.5990 - val_loss: 0.3422 - val_nse: 0.5841
Epoch 33/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.3210 - nse: 0.6176 - val_loss: 0.3378 - val_nse: 0.5893
Epoch 34/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 44ms/step - loss: 0.3623 - nse: 0.5998 - val_loss: 0.3403 - val_nse: 0.5864
Epoch 35/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.3417 - nse: 0.6023 - val_loss: 0.3347 - val_nse: 0.5931
Epoch 36/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.3072 - nse: 0.6277 - val_loss: 0.3354 - val_nse: 0.5923
Epoch 37/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.3252 - nse: 0.6109 - val_loss: 0.3371 - val_nse: 0.5903
Epoch 38/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 44ms/step - loss: 0.3266 - nse: 0.6137 - val_loss: 0.3362 - val_nse: 0.5914
Epoch 39/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 44ms/step - loss: 0.3760 - nse: 0.6012 - val_loss: 0.3307 - val_nse: 0.5980
Epoch 40/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.3555 - nse: 0.6171 - val_loss: 0.3337 - val_nse: 0.5944
Epoch 41/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.3413 - nse: 0.6182 - val_loss: 0.3299 - val_nse: 0.5990
Epoch 42/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.3031 - nse: 0.6286 - val_loss: 0.3260 - val_nse: 0.6037
Epoch 43/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2794 - nse: 0.6550 - val_loss: 0.3226 - val_nse: 0.6078
Epoch 44/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 44ms/step - loss: 0.3182 - nse: 0.6340 - val_loss: 0.3269 - val_nse: 0.6027
Epoch 45/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.3481 - nse: 0.6388 - val_loss: 0.3205 - val_nse: 0.6104
Epoch 46/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 44ms/step - loss: 0.3070 - nse: 0.6360 - val_loss: 0.3213 - val_nse: 0.6094
Epoch 47/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.3064 - nse: 0.6274 - val_loss: 0.3228 - val_nse: 0.6076
Epoch 48/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.3188 - nse: 0.6356 - val_loss: 0.3225 - val_nse: 0.6079
Epoch 49/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.3060 - nse: 0.6544 - val_loss: 0.3208 - val_nse: 0.6101
Epoch 50/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2810 - nse: 0.6509 - val_loss: 0.3135 - val_nse: 0.6189
Epoch 51/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.3195 - nse: 0.6434 - val_loss: 0.3158 - val_nse: 0.6162
Epoch 52/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2787 - nse: 0.6664 - val_loss: 0.3145 - val_nse: 0.6177
Epoch 53/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 44ms/step - loss: 0.2573 - nse: 0.6682 - val_loss: 0.3096 - val_nse: 0.6237
Epoch 54/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 44ms/step - loss: 0.2936 - nse: 0.6685 - val_loss: 0.3129 - val_nse: 0.6196
Epoch 55/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 44ms/step - loss: 0.3342 - nse: 0.6521 - val_loss: 0.3062 - val_nse: 0.6278
Epoch 56/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 44ms/step - loss: 0.2812 - nse: 0.6717 - val_loss: 0.3004 - val_nse: 0.6349
Epoch 57/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2733 - nse: 0.6611 - val_loss: 0.2987 - val_nse: 0.6370
Epoch 58/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2843 - nse: 0.6802 - val_loss: 0.3090 - val_nse: 0.6244
Epoch 59/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2356 - nse: 0.6917 - val_loss: 0.2939 - val_nse: 0.6428
Epoch 60/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2535 - nse: 0.6930 - val_loss: 0.2902 - val_nse: 0.6473
Epoch 61/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.3029 - nse: 0.6656 - val_loss: 0.2910 - val_nse: 0.6463
Epoch 62/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2932 - nse: 0.6817 - val_loss: 0.2934 - val_nse: 0.6433
Epoch 63/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2884 - nse: 0.6754 - val_loss: 0.2956 - val_nse: 0.6407
Epoch 64/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2852 - nse: 0.6735 - val_loss: 0.2824 - val_nse: 0.6567
Epoch 65/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 44ms/step - loss: 0.2446 - nse: 0.6946 - val_loss: 0.2831 - val_nse: 0.6558
Epoch 66/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 45ms/step - loss: 0.2725 - nse: 0.6934 - val_loss: 0.2853 - val_nse: 0.6532
Epoch 67/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.3083 - nse: 0.6766 - val_loss: 0.2801 - val_nse: 0.6596
Epoch 68/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 44ms/step - loss: 0.2306 - nse: 0.7054 - val_loss: 0.2790 - val_nse: 0.6609
Epoch 69/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2534 - nse: 0.7133 - val_loss: 0.2838 - val_nse: 0.6550
Epoch 70/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2551 - nse: 0.7079 - val_loss: 0.2661 - val_nse: 0.6766
Epoch 71/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 45ms/step - loss: 0.2553 - nse: 0.7036 - val_loss: 0.2653 - val_nse: 0.6775
Epoch 72/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2474 - nse: 0.7044 - val_loss: 0.2670 - val_nse: 0.6755
Epoch 73/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2611 - nse: 0.7095 - val_loss: 0.2668 - val_nse: 0.6757
Epoch 74/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2409 - nse: 0.7183 - val_loss: 0.2694 - val_nse: 0.6725
Epoch 75/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 45ms/step - loss: 0.2315 - nse: 0.7359 - val_loss: 0.2618 - val_nse: 0.6817
Epoch 76/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 44ms/step - loss: 0.2538 - nse: 0.7143 - val_loss: 0.2609 - val_nse: 0.6829
Epoch 77/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2388 - nse: 0.7149 - val_loss: 0.2586 - val_nse: 0.6856
Epoch 78/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2717 - nse: 0.7075 - val_loss: 0.2682 - val_nse: 0.6740
Epoch 79/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2068 - nse: 0.7292 - val_loss: 0.2633 - val_nse: 0.6799
Epoch 80/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2346 - nse: 0.7351 - val_loss: 0.2664 - val_nse: 0.6762
Epoch 81/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2164 - nse: 0.7326 - val_loss: 0.2672 - val_nse: 0.6752
Epoch 82/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2233 - nse: 0.7317 - val_loss: 0.2645 - val_nse: 0.6785
Epoch 83/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 44ms/step - loss: 0.2266 - nse: 0.7340 - val_loss: 0.2690 - val_nse: 0.6730
Epoch 84/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2153 - nse: 0.7290 - val_loss: 0.2689 - val_nse: 0.6731
Epoch 85/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 44ms/step - loss: 0.2261 - nse: 0.7414 - val_loss: 0.2686 - val_nse: 0.6735
Epoch 86/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2505 - nse: 0.7302 - val_loss: 0.2656 - val_nse: 0.6771
Epoch 87/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 44ms/step - loss: 0.2278 - nse: 0.7452 - val_loss: 0.2693 - val_nse: 0.6726
Epoch 88/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2373 - nse: 0.7376 - val_loss: 0.2673 - val_nse: 0.6751
Epoch 89/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2385 - nse: 0.7385 - val_loss: 0.2644 - val_nse: 0.6786
Epoch 90/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2183 - nse: 0.7559 - val_loss: 0.2671 - val_nse: 0.6753
Epoch 91/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2277 - nse: 0.7422 - val_loss: 0.2657 - val_nse: 0.6771
Epoch 92/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2754 - nse: 0.7065 - val_loss: 0.2509 - val_nse: 0.6950
Epoch 93/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2471 - nse: 0.7160 - val_loss: 0.2569 - val_nse: 0.6878
Epoch 94/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2243 - nse: 0.7460 - val_loss: 0.2782 - val_nse: 0.6618
Epoch 95/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 45ms/step - loss: 0.2109 - nse: 0.7520 - val_loss: 0.2676 - val_nse: 0.6747
Epoch 96/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 44ms/step - loss: 0.2050 - nse: 0.7554 - val_loss: 0.2601 - val_nse: 0.6838
Epoch 97/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2172 - nse: 0.7563 - val_loss: 0.2539 - val_nse: 0.6914
Epoch 98/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2417 - nse: 0.7351 - val_loss: 0.2627 - val_nse: 0.6807
Epoch 99/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 44ms/step - loss: 0.2077 - nse: 0.7401 - val_loss: 0.2605 - val_nse: 0.6833
Epoch 100/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 43ms/step - loss: 0.2515 - nse: 0.7350 - val_loss: 0.2611 - val_nse: 0.6827
valid_loss, valid_nse = rnn_model.evaluate(valid_ds)
valid_nse
1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 121ms/step - loss: 0.2509 - nse: 0.6950
0.6949852705001831
plt.figure()
plt.xlabel('Epoch')
plt.ylabel('Mean squared error')
plt.plot(history.epoch, np.array(history.history['loss']),label='Train Loss')
plt.plot(history.epoch, np.array(history.history['val_loss']),label = 'Val loss')
plt.legend()
<matplotlib.legend.Legend at 0x7e0d881c7950>
out = rnn_model.predict(test_ds)
1/1 ━━━━━━━━━━━━━━━━━━━━ 1s 564ms/step
out.shape
(3288, 1)
for x, y in test_ds.take(1):
yvals = y.numpy()
plt.plot(yvals,label="true")
plt.plot(out[:,0],label="prediction")
plt.legend()
<matplotlib.legend.Legend at 0x7e0cf03d7020>
Train an LSTM model#
from tensorflow.keras.initializers import Orthogonal
dropout_rate = 0.0
tf.random.set_seed(42) # ensures reproducibility
lstm_model = tf.keras.models.Sequential([
tf.keras.layers.LSTM(n_hidden, input_shape=[None, 5], return_sequences=False),
tf.keras.layers.Dropout(dropout_rate),
tf.keras.layers.Dense(1)
])
lstm_model.summary()
Model: "sequential_1"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩ │ lstm (LSTM) │ (None, 10) │ 640 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dropout (Dropout) │ (None, 10) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dense_1 (Dense) │ (None, 1) │ 11 │ └─────────────────────────────────┴────────────────────────┴───────────────┘
Total params: 651 (2.54 KB)
Trainable params: 651 (2.54 KB)
Non-trainable params: 0 (0.00 B)
early_stopping_cb = tf.keras.callbacks.EarlyStopping(
monitor="val_loss", patience=patience, restore_best_weights=True)
opt = tf.keras.optimizers.Adam(learning_rate=learning_rate)
lstm_model.compile(loss='mse', optimizer=opt, metrics=[NashSutcliffeEfficiency(scaler=target_scaler)])
history_lstm = lstm_model.fit(train_ds, validation_data=valid_ds, epochs=epochs,callbacks=[early_stopping_cb])
lstm_model.save(os.path.join(model_path,'LSTM_timeseries_model.keras'))
Epoch 1/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 3s 34ms/step - loss: 1.0360 - nse: -0.2738 - val_loss: 0.8328 - val_nse: -0.0124
Epoch 2/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.8614 - nse: 0.0108 - val_loss: 0.6761 - val_nse: 0.1781
Epoch 3/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.6929 - nse: 0.1742 - val_loss: 0.5764 - val_nse: 0.2993
Epoch 4/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 23ms/step - loss: 0.5599 - nse: 0.2813 - val_loss: 0.4968 - val_nse: 0.3962
Epoch 5/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.5378 - nse: 0.3731 - val_loss: 0.4296 - val_nse: 0.4778
Epoch 6/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.4391 - nse: 0.4639 - val_loss: 0.3952 - val_nse: 0.5196
Epoch 7/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.4061 - nse: 0.5293 - val_loss: 0.3599 - val_nse: 0.5625
Epoch 8/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.3466 - nse: 0.5747 - val_loss: 0.3341 - val_nse: 0.5938
Epoch 9/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.3191 - nse: 0.6177 - val_loss: 0.3146 - val_nse: 0.6175
Epoch 10/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.3282 - nse: 0.6301 - val_loss: 0.2999 - val_nse: 0.6355
Epoch 11/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.2688 - nse: 0.6659 - val_loss: 0.2915 - val_nse: 0.6456
Epoch 12/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.2815 - nse: 0.6677 - val_loss: 0.2788 - val_nse: 0.6611
Epoch 13/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.2851 - nse: 0.6728 - val_loss: 0.2720 - val_nse: 0.6693
Epoch 14/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.2554 - nse: 0.6918 - val_loss: 0.2645 - val_nse: 0.6784
Epoch 15/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.2609 - nse: 0.6929 - val_loss: 0.2559 - val_nse: 0.6890
Epoch 16/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.2472 - nse: 0.7027 - val_loss: 0.2518 - val_nse: 0.6939
Epoch 17/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.2755 - nse: 0.7021 - val_loss: 0.2431 - val_nse: 0.7045
Epoch 18/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.2488 - nse: 0.7063 - val_loss: 0.2408 - val_nse: 0.7073
Epoch 19/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.2450 - nse: 0.7221 - val_loss: 0.2322 - val_nse: 0.7178
Epoch 20/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.2574 - nse: 0.7175 - val_loss: 0.2266 - val_nse: 0.7245
Epoch 21/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.2344 - nse: 0.7296 - val_loss: 0.2254 - val_nse: 0.7260
Epoch 22/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1979 - nse: 0.7381 - val_loss: 0.2211 - val_nse: 0.7312
Epoch 23/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.2156 - nse: 0.7485 - val_loss: 0.2134 - val_nse: 0.7406
Epoch 24/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.2181 - nse: 0.7445 - val_loss: 0.2102 - val_nse: 0.7445
Epoch 25/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.2238 - nse: 0.7465 - val_loss: 0.2075 - val_nse: 0.7478
Epoch 26/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 24ms/step - loss: 0.1962 - nse: 0.7683 - val_loss: 0.2066 - val_nse: 0.7489
Epoch 27/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.1925 - nse: 0.7770 - val_loss: 0.2028 - val_nse: 0.7535
Epoch 28/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.1943 - nse: 0.7711 - val_loss: 0.1978 - val_nse: 0.7596
Epoch 29/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1752 - nse: 0.7829 - val_loss: 0.1941 - val_nse: 0.7640
Epoch 30/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1912 - nse: 0.7815 - val_loss: 0.1950 - val_nse: 0.7630
Epoch 31/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1837 - nse: 0.7863 - val_loss: 0.1865 - val_nse: 0.7733
Epoch 32/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1561 - nse: 0.8019 - val_loss: 0.1887 - val_nse: 0.7706
Epoch 33/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1848 - nse: 0.7969 - val_loss: 0.1828 - val_nse: 0.7778
Epoch 34/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1554 - nse: 0.8128 - val_loss: 0.1848 - val_nse: 0.7754
Epoch 35/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 20ms/step - loss: 0.1530 - nse: 0.8158 - val_loss: 0.1859 - val_nse: 0.7740
Epoch 36/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1674 - nse: 0.8083 - val_loss: 0.1723 - val_nse: 0.7905
Epoch 37/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1609 - nse: 0.8073 - val_loss: 0.1711 - val_nse: 0.7920
Epoch 38/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1434 - nse: 0.8255 - val_loss: 0.1683 - val_nse: 0.7955
Epoch 39/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 20ms/step - loss: 0.1510 - nse: 0.8266 - val_loss: 0.1759 - val_nse: 0.7861
Epoch 40/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.1562 - nse: 0.8163 - val_loss: 0.1604 - val_nse: 0.8050
Epoch 41/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1418 - nse: 0.8362 - val_loss: 0.1701 - val_nse: 0.7933
Epoch 42/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1438 - nse: 0.8338 - val_loss: 0.1597 - val_nse: 0.8059
Epoch 43/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1455 - nse: 0.8420 - val_loss: 0.1762 - val_nse: 0.7858
Epoch 44/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1331 - nse: 0.8471 - val_loss: 0.1770 - val_nse: 0.7848
Epoch 45/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.1313 - nse: 0.8467 - val_loss: 0.1588 - val_nse: 0.8069
Epoch 46/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1332 - nse: 0.8415 - val_loss: 0.1629 - val_nse: 0.8020
Epoch 47/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1327 - nse: 0.8571 - val_loss: 0.1639 - val_nse: 0.8008
Epoch 48/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 20ms/step - loss: 0.1227 - nse: 0.8565 - val_loss: 0.1612 - val_nse: 0.8041
Epoch 49/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1303 - nse: 0.8560 - val_loss: 0.1622 - val_nse: 0.8028
Epoch 50/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1236 - nse: 0.8605 - val_loss: 0.1543 - val_nse: 0.8124
Epoch 51/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1154 - nse: 0.8664 - val_loss: 0.1714 - val_nse: 0.7916
Epoch 52/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1253 - nse: 0.8608 - val_loss: 0.1624 - val_nse: 0.8026
Epoch 53/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1154 - nse: 0.8600 - val_loss: 0.1554 - val_nse: 0.8111
Epoch 54/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1208 - nse: 0.8598 - val_loss: 0.1581 - val_nse: 0.8078
Epoch 55/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1186 - nse: 0.8710 - val_loss: 0.1951 - val_nse: 0.7628
Epoch 56/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1188 - nse: 0.8647 - val_loss: 0.1540 - val_nse: 0.8128
Epoch 57/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1026 - nse: 0.8792 - val_loss: 0.1719 - val_nse: 0.7910
Epoch 58/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1073 - nse: 0.8754 - val_loss: 0.1501 - val_nse: 0.8175
Epoch 59/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1149 - nse: 0.8714 - val_loss: 0.1476 - val_nse: 0.8205
Epoch 60/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.1094 - nse: 0.8751 - val_loss: 0.1506 - val_nse: 0.8170
Epoch 61/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.0976 - nse: 0.8785 - val_loss: 0.1563 - val_nse: 0.8100
Epoch 62/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1082 - nse: 0.8816 - val_loss: 0.1526 - val_nse: 0.8145
Epoch 63/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.1047 - nse: 0.8834 - val_loss: 0.1450 - val_nse: 0.8238
Epoch 64/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1061 - nse: 0.8773 - val_loss: 0.1563 - val_nse: 0.8100
Epoch 65/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1069 - nse: 0.8824 - val_loss: 0.1422 - val_nse: 0.8271
Epoch 66/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.1015 - nse: 0.8872 - val_loss: 0.1429 - val_nse: 0.8263
Epoch 67/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 23ms/step - loss: 0.0842 - nse: 0.8808 - val_loss: 0.1601 - val_nse: 0.8054
Epoch 68/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1072 - nse: 0.8837 - val_loss: 0.1438 - val_nse: 0.8252
Epoch 69/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.0948 - nse: 0.8883 - val_loss: 0.1442 - val_nse: 0.8247
Epoch 70/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.1016 - nse: 0.8787 - val_loss: 0.1417 - val_nse: 0.8277
Epoch 71/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.0934 - nse: 0.8940 - val_loss: 0.1422 - val_nse: 0.8272
Epoch 72/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.0862 - nse: 0.8964 - val_loss: 0.1410 - val_nse: 0.8286
Epoch 73/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.0909 - nse: 0.8951 - val_loss: 0.1481 - val_nse: 0.8200
Epoch 74/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.0960 - nse: 0.8832 - val_loss: 0.1375 - val_nse: 0.8328
Epoch 75/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.0857 - nse: 0.8974 - val_loss: 0.1449 - val_nse: 0.8239
Epoch 76/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.0931 - nse: 0.8929 - val_loss: 0.1377 - val_nse: 0.8326
Epoch 77/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.0870 - nse: 0.8996 - val_loss: 0.1488 - val_nse: 0.8191
Epoch 78/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.0813 - nse: 0.8946 - val_loss: 0.1331 - val_nse: 0.8382
Epoch 79/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.0941 - nse: 0.8914 - val_loss: 0.1323 - val_nse: 0.8392
Epoch 80/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.0906 - nse: 0.8945 - val_loss: 0.1391 - val_nse: 0.8309
Epoch 81/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.0861 - nse: 0.8971 - val_loss: 0.1375 - val_nse: 0.8329
Epoch 82/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.0924 - nse: 0.8953 - val_loss: 0.1374 - val_nse: 0.8330
Epoch 83/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.0824 - nse: 0.9040 - val_loss: 0.1381 - val_nse: 0.8321
Epoch 84/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.0847 - nse: 0.9020 - val_loss: 0.1325 - val_nse: 0.8389
Epoch 85/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.0851 - nse: 0.9011 - val_loss: 0.1510 - val_nse: 0.8164
Epoch 86/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.0866 - nse: 0.8902 - val_loss: 0.1376 - val_nse: 0.8327
Epoch 87/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.0884 - nse: 0.8924 - val_loss: 0.1407 - val_nse: 0.8290
Epoch 88/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.0834 - nse: 0.9021 - val_loss: 0.1479 - val_nse: 0.8202
Epoch 89/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.0845 - nse: 0.8997 - val_loss: 0.1309 - val_nse: 0.8409
Epoch 90/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.0788 - nse: 0.8986 - val_loss: 0.1312 - val_nse: 0.8406
Epoch 91/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.0895 - nse: 0.9011 - val_loss: 0.1367 - val_nse: 0.8338
Epoch 92/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.0849 - nse: 0.8980 - val_loss: 0.1410 - val_nse: 0.8286
Epoch 93/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.0781 - nse: 0.9066 - val_loss: 0.1348 - val_nse: 0.8361
Epoch 94/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.0786 - nse: 0.9034 - val_loss: 0.1313 - val_nse: 0.8404
Epoch 95/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 24ms/step - loss: 0.0776 - nse: 0.9064 - val_loss: 0.1284 - val_nse: 0.8439
Epoch 96/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.0768 - nse: 0.9068 - val_loss: 0.1327 - val_nse: 0.8386
Epoch 97/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.0778 - nse: 0.9094 - val_loss: 0.1298 - val_nse: 0.8423
Epoch 98/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.0880 - nse: 0.9085 - val_loss: 0.1297 - val_nse: 0.8423
Epoch 99/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.0756 - nse: 0.9124 - val_loss: 0.1281 - val_nse: 0.8442
Epoch 100/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.0713 - nse: 0.9075 - val_loss: 0.1411 - val_nse: 0.8285
plt.figure()
plt.xlabel('Epoch')
plt.ylabel('Mean squared error')
plt.plot(history.epoch, np.array(history.history['loss']),label='Train Loss - RNN')
plt.plot(history.epoch, np.array(history.history['val_loss']),label = 'Val loss - RNN')
plt.plot(history_lstm.epoch, np.array(history_lstm.history['loss']),label='Train Loss - LSTM')
plt.plot(history_lstm.epoch, np.array(history_lstm.history['val_loss']),label = 'Val loss - LSTM')
plt.legend()
<matplotlib.legend.Legend at 0x7e0cf0290140>
out_lstm = lstm_model.predict(test_ds)
1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 310ms/step
plt.plot(yvals,label="true")
plt.plot(out_lstm[:,0],label="prediction - LSTM")
plt.legend()
<matplotlib.legend.Legend at 0x7e0cf03e7290>
Train a GRU model#
The GRU sits between the RNN and the LSTM in complexity. (Keras reports slightly more parameters than the classic GRU formula from the notes because its default implementation, reset_after=True, carries a second set of bias vectors.)
tf.random.set_seed(42) # ensures reproducibility
gru_model = tf.keras.Sequential([
tf.keras.layers.GRU(n_hidden, return_sequences=False, input_shape=[None, 5]),
tf.keras.layers.Dense(1)
])
gru_model.summary()
Model: "sequential_2"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩ │ gru (GRU) │ (None, 10) │ 510 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dense_2 (Dense) │ (None, 1) │ 11 │ └─────────────────────────────────┴────────────────────────┴───────────────┘
Total params: 521 (2.04 KB)
Trainable params: 521 (2.04 KB)
Non-trainable params: 0 (0.00 B)
early_stopping_cb = tf.keras.callbacks.EarlyStopping(
monitor="val_loss", patience=patience, restore_best_weights=True)
opt = tf.keras.optimizers.Adam(learning_rate=learning_rate)
gru_model.compile(loss='mse', optimizer=opt, metrics=[NashSutcliffeEfficiency(scaler=target_scaler)])
history_gru = gru_model.fit(train_ds, validation_data=valid_ds, epochs=epochs,callbacks=[early_stopping_cb])
gru_model.save(os.path.join(model_path,'GRU_timeseries_model.keras'))
Epoch 1/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 2s 34ms/step - loss: 0.8992 - nse: -0.0068 - val_loss: 0.6888 - val_nse: 0.1628
Epoch 2/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.6522 - nse: 0.1597 - val_loss: 0.5882 - val_nse: 0.2850
Epoch 3/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.6388 - nse: 0.2456 - val_loss: 0.5335 - val_nse: 0.3515
Epoch 4/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.5446 - nse: 0.3279 - val_loss: 0.4881 - val_nse: 0.4066
Epoch 5/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 23ms/step - loss: 0.5215 - nse: 0.3803 - val_loss: 0.4516 - val_nse: 0.4510
Epoch 6/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.5043 - nse: 0.4261 - val_loss: 0.4241 - val_nse: 0.4845
Epoch 7/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.4854 - nse: 0.4729 - val_loss: 0.4001 - val_nse: 0.5137
Epoch 8/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.4374 - nse: 0.4975 - val_loss: 0.3762 - val_nse: 0.5427
Epoch 9/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.3837 - nse: 0.5309 - val_loss: 0.3568 - val_nse: 0.5662
Epoch 10/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.3680 - nse: 0.5786 - val_loss: 0.3421 - val_nse: 0.5841
Epoch 11/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.3329 - nse: 0.5966 - val_loss: 0.3273 - val_nse: 0.6021
Epoch 12/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.3308 - nse: 0.6235 - val_loss: 0.3165 - val_nse: 0.6153
Epoch 13/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 25ms/step - loss: 0.3448 - nse: 0.6198 - val_loss: 0.3035 - val_nse: 0.6311
Epoch 14/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.3355 - nse: 0.6316 - val_loss: 0.2954 - val_nse: 0.6409
Epoch 15/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.3332 - nse: 0.6484 - val_loss: 0.2889 - val_nse: 0.6488
Epoch 16/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.2544 - nse: 0.6874 - val_loss: 0.2803 - val_nse: 0.6592
Epoch 17/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.3095 - nse: 0.6629 - val_loss: 0.2719 - val_nse: 0.6695
Epoch 18/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.2789 - nse: 0.6886 - val_loss: 0.2655 - val_nse: 0.6772
Epoch 19/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.2814 - nse: 0.6820 - val_loss: 0.2580 - val_nse: 0.6864
Epoch 20/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.2579 - nse: 0.6932 - val_loss: 0.2532 - val_nse: 0.6922
Epoch 21/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.2421 - nse: 0.7066 - val_loss: 0.2446 - val_nse: 0.7027
Epoch 22/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.2456 - nse: 0.7090 - val_loss: 0.2359 - val_nse: 0.7132
Epoch 23/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.2559 - nse: 0.7222 - val_loss: 0.2319 - val_nse: 0.7181
Epoch 24/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.2394 - nse: 0.7241 - val_loss: 0.2242 - val_nse: 0.7275
Epoch 25/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.2448 - nse: 0.7293 - val_loss: 0.2168 - val_nse: 0.7364
Epoch 26/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.2293 - nse: 0.7412 - val_loss: 0.2199 - val_nse: 0.7327
Epoch 27/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.2287 - nse: 0.7522 - val_loss: 0.2077 - val_nse: 0.7476
Epoch 28/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1920 - nse: 0.7781 - val_loss: 0.2090 - val_nse: 0.7459
Epoch 29/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1989 - nse: 0.7684 - val_loss: 0.1982 - val_nse: 0.7591
Epoch 30/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.2077 - nse: 0.7708 - val_loss: 0.1957 - val_nse: 0.7621
Epoch 31/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1845 - nse: 0.7778 - val_loss: 0.1952 - val_nse: 0.7627
Epoch 32/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.1874 - nse: 0.7863 - val_loss: 0.1921 - val_nse: 0.7664
Epoch 33/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1835 - nse: 0.7891 - val_loss: 0.1853 - val_nse: 0.7748
Epoch 34/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1932 - nse: 0.7890 - val_loss: 0.1953 - val_nse: 0.7626
Epoch 35/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1635 - nse: 0.8026 - val_loss: 0.1852 - val_nse: 0.7749
Epoch 36/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1475 - nse: 0.8028 - val_loss: 0.1907 - val_nse: 0.7682
Epoch 37/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1573 - nse: 0.8165 - val_loss: 0.1803 - val_nse: 0.7808
Epoch 38/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1786 - nse: 0.8016 - val_loss: 0.1852 - val_nse: 0.7749
Epoch 39/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1612 - nse: 0.8078 - val_loss: 0.1814 - val_nse: 0.7795
Epoch 40/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1606 - nse: 0.8114 - val_loss: 0.1777 - val_nse: 0.7840
Epoch 41/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1563 - nse: 0.8131 - val_loss: 0.1964 - val_nse: 0.7612
Epoch 42/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 23ms/step - loss: 0.1514 - nse: 0.8189 - val_loss: 0.1799 - val_nse: 0.7813
Epoch 43/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1634 - nse: 0.8172 - val_loss: 0.1789 - val_nse: 0.7825
Epoch 44/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1413 - nse: 0.8227 - val_loss: 0.1766 - val_nse: 0.7853
Epoch 45/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1597 - nse: 0.8226 - val_loss: 0.1728 - val_nse: 0.7900
Epoch 46/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.1531 - nse: 0.8220 - val_loss: 0.1777 - val_nse: 0.7840
Epoch 47/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.1622 - nse: 0.8225 - val_loss: 0.1735 - val_nse: 0.7891
Epoch 48/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1521 - nse: 0.8337 - val_loss: 0.1788 - val_nse: 0.7826
Epoch 49/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1489 - nse: 0.8348 - val_loss: 0.1748 - val_nse: 0.7875
Epoch 50/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step - loss: 0.1437 - nse: 0.8345 - val_loss: 0.1764 - val_nse: 0.7855
Epoch 51/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1378 - nse: 0.8412 - val_loss: 0.1735 - val_nse: 0.7891
Epoch 52/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1439 - nse: 0.8308 - val_loss: 0.1753 - val_nse: 0.7869
Epoch 53/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1327 - nse: 0.8515 - val_loss: 0.1731 - val_nse: 0.7896
Epoch 54/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1451 - nse: 0.8359 - val_loss: 0.1758 - val_nse: 0.7863
Epoch 55/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1511 - nse: 0.8441 - val_loss: 0.1748 - val_nse: 0.7875
Epoch 56/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1240 - nse: 0.8422 - val_loss: 0.1768 - val_nse: 0.7851
Epoch 57/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1335 - nse: 0.8497 - val_loss: 0.1761 - val_nse: 0.7859
Epoch 58/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1307 - nse: 0.8472 - val_loss: 0.1736 - val_nse: 0.7890
Epoch 59/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1141 - nse: 0.8608 - val_loss: 0.1781 - val_nse: 0.7835
Epoch 60/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1300 - nse: 0.8512 - val_loss: 0.1772 - val_nse: 0.7846
Epoch 61/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 23ms/step - loss: 0.1206 - nse: 0.8537 - val_loss: 0.1741 - val_nse: 0.7884
Epoch 62/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1207 - nse: 0.8594 - val_loss: 0.1899 - val_nse: 0.7692
Epoch 63/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1133 - nse: 0.8674 - val_loss: 0.2028 - val_nse: 0.7535
Epoch 64/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1269 - nse: 0.8491 - val_loss: 0.2012 - val_nse: 0.7554
Epoch 65/100
20/20 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - loss: 0.1081 - nse: 0.8713 - val_loss: 0.1840 - val_nse: 0.7764
plt.figure()
plt.xlabel('Epoch')
plt.ylabel('Mean squared error')
plt.plot(history.epoch, np.array(history.history['loss']),label='Train Loss - RNN')
plt.plot(history.epoch, np.array(history.history['val_loss']),label = 'Val loss - RNN')
plt.plot(history_lstm.epoch, np.array(history_lstm.history['loss']),label='Train Loss - LSTM')
plt.plot(history_lstm.epoch, np.array(history_lstm.history['val_loss']),label = 'Val loss - LSTM')
plt.plot(history_gru.epoch, np.array(history_gru.history['loss']),label='Train Loss - GRU')
plt.plot(history_gru.epoch, np.array(history_gru.history['val_loss']),label = 'Val loss - GRU')
plt.legend()
<matplotlib.legend.Legend at 0x7e0ce001ce90>
out_gru = gru_model.predict(test_ds)
1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 285ms/step
plt.plot(yvals,label="true")
plt.plot(out_lstm[:,0],label="prediction - LSTM")
plt.plot(out_gru[:,0],label="prediction - GRU")
plt.plot(out[:,0],label="prediction - RNN")
plt.legend()
<matplotlib.legend.Legend at 0x7e0ccc545b80>
