Skip to content

Commit

Permalink
Try to make baseline model less prone to overfitting
Browse files Browse the repository at this point in the history
Training history showed that (proper) val metrics (evaluated on a
different synthesis tree) behave better with this model version. A
clear val_loss minimum is now seen (and I'm overtraining the model).
  • Loading branch information
iburakov committed Aug 23, 2023
1 parent 293a7b9 commit fbf96c5
Showing 1 changed file with 5 additions and 5 deletions.
10 changes: 5 additions & 5 deletions ml/synthesis/src/components/model_generation/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,16 @@ def create_baseline_model(input_shape) -> Model:
model = Sequential(
[
layers.InputLayer(input_shape=input_shape),
layers.Dense(128, activation="relu"),
layers.Dense(64, activation="relu"),
layers.Dense(64, activation="relu"),
layers.Dense(32, activation="relu"),
layers.Dense(64, activation="relu", kernel_regularizer="l2"),
layers.Dropout(0.5),
layers.Dense(16, activation="relu"),
layers.Dense(16, activation="relu"),
layers.Dense(1),
],
)

model.compile(
optimizer=optimizers.Adam(learning_rate=1e-3),
optimizer=optimizers.Adam(learning_rate=1e-4),
loss="mse",
metrics=["mae"],
)
Expand Down

0 comments on commit fbf96c5

Please sign in to comment.