Accessing Rubin Data Preview 2 (DP2)

Accessing Rubin Data Preview 2 (DP2)#

In this tutorial, we will:

  • access Rubin’s Data Preview 2 with LSDB

    • at RSP (Rubin Science Platform), based in the USA and the UK

Introduction#

Prerequisites#

In order to access Rubin data, you must be a Rubin data rights holder.

1. Accessing the data on Rubin Science Platform (RSP)#

1.1 Prepare your RSP container#

  • Visit https://data.lsst.cloud, unless you are accessing RSP through UK IDAC participation program - in that case, visit https://rsp.lsst.ac.uk.

  • Log in using your identity provider.

  • Once in, you will see Portal, Notebooks, and APIs. Choose Notebooks.

  • When it asks you what container to start, choose either “Recommended” or one of the latest weekly releases on the left, and “Large” on the right.

    • At the time of this writing, “Recommended” comes with an older version of lsdb - run the cell below to see if you’ll need to target a weekly release instead.

  • Once this has started, create a new notebook.

1.1.1 Ensure your notebook kernel has the right version of lsdb#

Make sure you’ve got at least version 0.10.1 of lsdb. Try the following.

[ ]:

%pip list | grep -E 'lsdb|hats'

[ ]:

# Or for even more detail:

import lsdb

lsdb.show_versions()

If the above shows a version that is very old, we suggest using RSP with a latest weekly release rather than the “Recommended” build.

1.2. Create a Dask Client#

When working on RSP, with a Large container you have access to 32 GB of RAM. To ensure each worker has enough memory to work with the data, we recommend using 4 workers with 1 thread each, and memory limit of “auto” (which will divide the available memory across the workers). Dask also will sometimes spill to disk when it needs to, so we recommend setting the local directory to the /deleted-sundays directory, which is a large scratch space available on RSP that will be cleared every Sunday. You can set up your Dask client with the following code:

[ ]:

# Dask puts out more advisory logging than we care for in this tutorial.
# It takes some doing to quiet all of it, but this recipe works.

import dask
import os

dask.config.set({"logging.distributed": "critical"})

import logging

# This also has to be done, for the above to be effective
logger = logging.getLogger("distributed")
logger.setLevel(logging.CRITICAL)

import warnings

# Finally, suppress the specific warning about Dask dashboard port usage
warnings.filterwarnings("ignore", message="Port 8787 is already in use.")

[ ]:

from dask.distributed import Client

client = Client(
    n_workers=4,
    threads_per_worker=1,
    memory_limit="auto",
    local_directory=f"/deleted-sundays/{os.environ.get('USER', 'dask_scratch')}",
)
client

Your Dask dashboard will be accessible at https://{username}.nb.data.lsst.cloud/nb/user/{username}/proxy/{port}/status.

1.3 Opening a Catalog#

The data is divided into objects and dia_objects. Let’s open both catalogs:

[ ]:

from upath import UPath

base_path = UPath("/rubin/lsdb_data/dp2")

object_cat = lsdb.open_catalog(base_path / "object_collection")
dia_object_cat = lsdb.open_catalog(base_path / "dia_object_collection")

[ ]:

object_cat

[ ]:

dia_object_cat

You’ve accessed the data! You can see the schema of both catalogs.

To learn how to use this data, see Using Rubin Data and follow the lsdb quickstart at Getting Started.

If you have questions about data from the Rubin Observatory, the best place to ask is https://community.lsst.org in the LSST Data Products category.

About#

Authors: Neven Caplar, Sandro Campos, Olivia Lynn

Last updated on: July 17, 2026

Last run: Never, waiting for DP2 release

If you use lsdb for published research, please cite following instructions.