{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "\n# Example 3: Adding Regularization and Penalty Terms to Fitting\n\nThis example is nearly identical to Example 2, however we\nuse a more sophisticated loss function, introducing an additional first-order\npenalty term. The previous synthetic fit relied only on MSE loss and a second-order penalty.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Import packages\n\nFirst, we import the packages we need for this example. Additionally, we choose to use\nCUDA if it is available.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "import torch\nimport torch.nn as nn\n\nfrom drdmannturb.parameters import (\n LossParameters,\n NNParameters,\n PhysicalParameters,\n ProblemParameters,\n)\nfrom drdmannturb.spectra_fitting import CalibrationProblem, OnePointSpectraDataGenerator\n\ndevice = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n\nif torch.cuda.is_available():\n torch.set_default_tensor_type(\"torch.cuda.FloatTensor\")\n\nzref = 40 # reference height\nustar = 1.773 # friction velocity\n\n# Scales associated with Kaimal spectrum\nL = 0.59 * zref # length scale\nGamma = 3.9 # time scale\nsigma = 3.2 * ustar**2.0 / zref ** (2.0 / 3.0) # magnitude (\u03c3 = \u03b1\u03f5^{2/3})\n\nprint(f\"Physical Parameters: {L,Gamma,sigma}\")\n\nk1 = torch.logspace(-1, 2, 20) / zref" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "%%\nNow, we construct our ``CalibrationProblem``.\n\nCompared to Example 2, we are instead using GELU\nactivations and will train for fewer epochs. The more interesting difference\nis that we will have activated a first order term in the loss function by passing\n``alpha_pen1`` a value in the ``LossParameters`` constructor.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "pb = CalibrationProblem(\n nn_params=NNParameters(\n nlayers=2,\n hidden_layer_sizes=[10, 10],\n activations=[nn.GELU(), nn.GELU()],\n ),\n prob_params=ProblemParameters(nepochs=5),\n loss_params=LossParameters(alpha_pen2=1.0, alpha_pen1=1.0e-5, beta_reg=2e-4),\n phys_params=PhysicalParameters(\n L=L, Gamma=Gamma, sigma=sigma, ustar=ustar, domain=k1\n ),\n logging_directory=\"runs/synthetic_3term\",\n device=device,\n)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In the following cell, we construct our $k_1$ data points grid and\ngenerate the values. ``Data`` will be a tuple ``(, )``.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "Data = OnePointSpectraDataGenerator(data_points=k1, zref=zref, ustar=ustar).Data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Calibration\nNow, we fit our model. ``CalibrationProblem.calibrate`` takes the tuple ``Data``\nwhich we just constructed and performs a typical training loop.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "optimal_parameters = pb.calibrate(data=Data)\n\npb.print_calibrated_params()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Plotting\n``DRDMannTurb`` offers built-in plotting utilities and Tensorboard integration\nwhich make visualizing results and various aspects of training performance\nvery simple. The training logs can be accessed from the logging directory\nwith Tensorboard utilities, but we also provide a simple internal utility for a single\ntraining log plot.\n\nThe following will plot our fit.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "pb.plot()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This plots out the loss function terms as specified, each multiplied by the\nrespective coefficient hyperparameter.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "pb.plot_losses(run_number=0)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.16" } }, "nbformat": 4, "nbformat_minor": 0 }