Add Neptune to your code#

In your Python script, import the Neptune client library:
```
import neptune
```

To begin tracking your ML experiment, initialize a Neptune run.

The minimal invocation is neptune.init_run(), but you can use a number of customization options:

import neptune

run = neptune.init_run(
    project="ml-team/classification", # (1)!
    name="awesome-woodpecker", # (2)!
    tags=["maskRCNN", "finetune"], # (3)!
    source_files=["**/*.py", "config.yaml"], # (4)!
    dependencies="infer", # (5)!
    monitoring_namespace="monitoring", # (6)!
)

Points to a Neptune workspace and project. We recommend setting the project name as an environment variable.
Sets a custom name, which you can use as a human-friendly ID.

To display it in the app, add sys/name as a column.

You can also edit the name in the run information view ( → Run information).
Tags the run for easier categorization. Learn more: Add tags
Specifies which source files to log for the run. By default, Neptune logs the script that was executed. Learn more: Log source code
Generates a requirements.txt file based on dependencies installed in the environment. Learn more: Track dependencies
Optional, but recommended. See Best practices: Monitoring system metrics.

Show full init_run() parameters list

See in API reference: neptune.init_run()

Name	Type	Default	Description
`project`	`str`, optional	`None`	Name of a project in the form `workspace-name/project-name`. If `None`, the value of the `NEPTUNE_PROJECT` environment variable is used.
`api_token`	`str`, optional	`None`	Your Neptune API token (or a service account's API token). If `None`, the value of the `NEPTUNE_API_TOKEN` environment variable is used. To keep your token secure, avoid placing it in source code. Instead, save it as an environment variable.
`with_id`	`str`, optional	`None`	The Neptune identifier of an existing run to resume, such as "CLS-11". The identifier is stored in the object's `sys/id` field. If omitted or `None` is passed, a new tracked run is created.
`custom_run_id`	`str`, optional	`None`	A unique identifier that can be used to log metadata to a single run from multiple locations. Max length: 36 characters. If `None` and the `NEPTUNE_CUSTOM_RUN_ID` environment variable is set, Neptune will use that as the `custom_run_id` value. For details, see Set custom run ID.
`mode`	`str`, optional	`async`	Connection mode in which the logging will work. Possible values are `async`, `sync`, `offline`, `read-only`, and `debug`. If you leave it out, the value of the `NEPTUNE_MODE` environment variable is used. If that's not set, the default `async` is used.
`name`	`str`, optional	Neptune ID	Custom name for the run. You can use it as a human-readable ID and add it as a column in the experiments table (`sys/name`). If left empty, once the run is synchronized with the server, Neptune sets the auto-generated identifier (`sys/id`) as the name.
`description`	`str`, optional	`""`	Editable description of the run. You can add it as a column in the experiments table (`sys/description`).
`tags`	`list`, optional	`[]`	Must be a list of `str` which represent the tags for the run. You can edit them after run is created, either in the run information or experiments table.
`source_files`	`list` or `str`, optional	`None`	List of source files to be uploaded. Must be list of `str` or a single `str`. Uploaded sources are displayed in the Source code section of the run. If `None` is passed, the Python file from which the run was created will be uploaded. When resuming a run, no file will be uploaded by default. Pass an empty list (`[]`) to upload no files. Unix style pathname pattern expansion is supported. For example, you can pass `".py"` to upload all Python source files from the current directory. Paths of uploaded files are resolved relative to the calculated common root of all uploaded source files. For recursion lookup, use `"**/.py"` (for Python `3.5` and later). For details, see the glob library.
`capture_stdout`	`Boolean`, optional	`True`	Whether to log the standard output stream. Is logged in the `monitoring` namespace.
`capture_stderr`	`Boolean`, optional	`True`	Whether to log the standard error stream. Is logged in the `monitoring` namespace.
`capture_hardware_metrics`	`Boolean`, optional	`True`	Whether to track hardware consumption (CPU, GPU, memory utilization). Logged in the `monitoring` namespace.
`fail_on_exception`	`Boolean`, optional	`True`	If an uncaught exception occurs, whether to set run's `Failed` state to `True`.
`monitoring_namespace`	`str`, optional	`"monitoring"`	Namespace inside which all monitoring logs will be stored.
`flush_period`	`float`, optional	`5` (seconds)	In asynchronous (default) connection mode, how often Neptune should trigger disk flushing.
`proxies`	`dict`, optional	`None`	Argument passed to HTTP calls made via the Requests library. For details on proxies, see the Requests documentation.
`capture_traceback`	`Boolean`, optional	`True`	In case of an exception, whether to log the traceback of the run.
`git_ref`	`GitRef` or `Boolean`	`None`	`GitRef` object containing information about the Git repository path. If `None`, Neptune looks for a repository in the path of the script that is executed. To specify a different location, set to `GitRef(repository_path="path/to/repo")`. To turn off Git tracking for the run, set to `GitRef.DISABLED` or `False`. For examples, see Logging Git info.
`dependencies`	`str`, optional	`None`	Tracks environment requirements. If you pass `"infer"` to this argument, Neptune logs dependencies installed in the current environment. You can also pass a path to your dependency file directly. If left empty, no dependency file is uploaded.
`async_lag_callback`	`NeptuneObjectCallback`, optional	`None`	Custom callback function which is called if the lag between a queued operation and its synchronization with the server exceeds the duration defined by `async_lag_threshold`. The callback should take a `Run` object as the argument and can contain any custom code, such as calling `stop()` on the object. Note: Instead of using this argument, you can use Neptune's default callback by setting the `NEPTUNE_ENABLE_DEFAULT_ASYNC_LAG_CALLBACK` environment variable to `TRUE`.
`async_lag_threshold`	`float`, optional	`1800.0` (seconds)	Duration between the queueing and synchronization of an operation. If a lag callback (default callback enabled via environment variable or custom callback passed to the `async_lag_callback` argument) is enabled, the callback is called when this duration is exceeded.
`async_no_progress_callback`	`NeptuneObjectCallback`, optional	`None`	Custom callback function which is called if there has been no synchronization progress whatsoever for the duration defined by `async_no_progress_threshold`. The callback should take a `Run` object as the argument and can contain any custom code, such as calling `stop()` on the object. Note: Instead of using this argument, you can use Neptune's default callback by setting the `NEPTUNE_ENABLE_DEFAULT_ASYNC_NO_PROGRESS_CALLBACK` environment variable to `TRUE`.
`async_no_progress_threshold`	`float`, optional	`300.0` (seconds)	For how long there has been no synchronization progress. If a no-progress callback (default callback enabled via environment variable or custom callback passed to the `async_no_progress_callback` argument) is enabled, the callback is called when this duration is exceeded.

Log experiment tracking metadata in a structure of your choice:

params = {
    "max_epochs": 10,
    "optimizer": "Adam",
    "dropout": 0.2,
}
run["parameters"] = params

When you're done, call stop() to end the connection and sync the remaining data:
```
run.stop()
```

Sample output

[neptune] [info ] Neptune initialized. Open in the app: https://app.neptune.ai/workspace/project/e/RUN-1

Example#

Below is a high-level example of how you can plug Neptune into a typical model-training flow.

Start the tracking#

In your model training script, import Neptune and initialize a run:

run = neptune.init_run() # (1)!

We recommend saving your API token and project name as environment variables.

If needed, you can pass them as arguments when initializing Neptune:
```
neptune.init_run(
    project="workspace-name/project-name",
    api_token="YourNeptuneApiToken",
)
```

Log hyperparameters#

Define some hyperparameters to track for the experiment and log them to the run object:

parameters = {
    "dense_units": 128,
    "activation": "relu",
    "dropout": 0.23,
    "learning_rate": 0.15,
    "batch_size": 64,
    "n_epochs": 30,
}
run["model/parameters"] = parameters

You can update or add new entries later in the code:

# Add additional parameters
run["model/parameters/seed"] = RANDOM_SEED

# Update parameters. For example, after triggering early stopping
run["model/parameters/n_epochs"] = epoch

Log training metrics#

Track the training process by logging your training metrics. Use the append() method for a series of values:

for epoch in range(parameters["n_epochs"]):
    ...  # My training loop

    run["train/epoch/loss"].append(loss)
    run["train/epoch/accuracy"].append(acc)

Batching tip

To optimize batching when creating multiple series fields in a single statement, iterate through the fields in the outer loop and the values in the inner loop:

Improves batching

for field in range(500):
    for i in range(1000):
        run[f"my_metric_{field}"].append(i)

Doesn't batch well

for i in range(1000):
    for field in range(500):
        run[f"my_metric_{field}"].append(i)

Or use one of our ready-made integrations:

PyTorch Lightning🤗 TransformersTensorFlow/Kerasscikit-learn

from lightning.pytorch.loggers import NeptuneLogger

neptune_logger = NeptuneLogger()

trainer = Trainer(
    ...
    logger=neptune_logger,
)

training_args = TrainingArguments(
    ...
    report_to="neptune", # (1)!
)

trainer = Trainer(
    model,
    training_args,
    ...
)

trainer.train()

Requires having your Neptune credentials as environment variables. For help, see Set credentials.

from neptune.integrations.tensorflow_keras import NeptuneCallback

model.fit(
    x_train,
    y_train,
    callbacks=[NeptuneCallback(run=run)],
)

import neptune.integrations.sklearn as npt_utils

run["cls_summary"] = npt_utils.create_classifier_summary(
    gbc, X_train, X_test, y_train, y_test
)

run["rfr_summary"] = npt_utils.create_regressor_summary(
    rfr, X_train, X_test, y_train, y_test
)

run["kmeans_summary"] = npt_utils.create_kmeans_summary(
    km, X, n_clusters=17
)

You can use Neptune with any machine learning framework.

If you use a framework that supports logging (most of them do) you don't need to write the logging code yourself. The Neptune integration takes care of tracking all the training metrics.

For details, see Integrations.

Log evaluation results#

Metrics#

Assign the metrics to a namespace and field of your choice:

run["evaluation/accuracy"] = eval_acc
run["evaluation/loss"] = eval_loss

Using the snippet above, both evaluation metrics will be logged in the same evaluation namespace.

Charts#

You can log plots and charts with the upload() method.

A plot object is converted to an image file, but you can also upload images from the local disk.

MatplotlibFrom disk

import matplotlib.pyplot as plt
from scikitplot.metrics import plot_roc, plot_precision_recall

fig, ax = plt.subplots()
plot_roc(y_test, y_pred_proba, ax=ax)

run["evaluation/ROC"].upload(fig)

fig, ax = plt.subplots()
plot_precision_recall(y_test, y_pred_proba, ax=ax)

run["evaluation/precision-recall"].upload(fig)

run["evaluation/ROC"].upload("roc.png")
run["evaluation/precision-recall"].upload("prec-recall.jpg")

Sample predictions#

The following snippet logs sample predictions as a series of labeled images:

for image, predicted_label, probabilites in sample_predictions:

    description = "\n".join(
        [f"class {label}: {prob}" for label, prob in probabilites]
    )

    run["evaluation/predictions"].append(
        image,
        name=predicted_label,
        description=description,
    )

You can upload tabular data as a pandas DataFrame, then inspect it as an interactive table in the app:

import pandas as pd

df = pd.DataFrame(
    data={
        "y_test": y_test,
        "y_pred": y_pred,
        "y_pred_probability": y_pred_proba.max(axis=1),
    }
)

run["evaluation/predictions"].upload(File.as_html(df))

Upload relevant files#

You can upload any binary file from disk with the upload() method.

If your model is saved as multiple files, you can upload a whole folder as a FileSet with upload_files().

From diskAs a pickle

torch.save(net.state_dict(), "model.pt")

run["model/saved_model"].upload("model.pt")

from neptune.types import File

run["model/pickled_model"].upload(File.as_pickle(model_object))

Track artifact versions#

Instead of uploading file contents, you can track their metadata only.

run["dataset/train"].track_files("./datasets/train/images")

For details, see Track artifacts.

Explore results#

Once you're done logging, end the run with the stop() method:

run.stop()

Next, run your script and follow the link to explore your metadata in Neptune.

If Neptune can't find your project name or API token

As a best practice, you should save your Neptune API token and project name as environment variables:

export NEPTUNE_API_TOKEN="h0dHBzOi8aHR0cHM6Lkc78ghs74kl0jv...Yh3Kb8"

export NEPTUNE_PROJECT="ml-team/classification"

Alternatively, you can pass the information when using a function that takes api_token and project as arguments:

run = neptune.init_run(
    api_token="h0dHBzOi8aHR0cHM6Lkc78ghs74kl0jv...Yh3Kb8", # (1)!
    project="ml-team/classification", # (2)!
)

In the bottom-left corner, expand the user menu and select Get my API token.
You can copy the path from the project details ( → Details & privacy).

If you haven't registered, you can log anonymously to a public project:

api_token=neptune.ANONYMOUS_API_TOKEN
project="common/quickstarts"

Make sure not to publish sensitive data through your code!

For a practical example, you can check out the source code that was used to log a Neptune showcase run.

See example code in Neptune More examples on GitHub