Overview
Multi-Scrobbler can bulk import your Play (scrobble) history and export it to your Clients.
This page takes you through one example from start to end. You import your Last.fm history, export it to a Client, and then clean up. Each step explains the parts of Multi-Scrobbler that it uses.
1. Create a Job
Section titled “1. Create a Job”A Job is the task that moves your Plays. A Job has two parts:
- An Import. It adds Plays to a Mirror.
- An optional Export. It sends the Plays in that Mirror to one or more Clients.
Each part runs on its own. You can pause, stop, and resume each part, so you can do the work at your own pace.
To start, open Menu -> Import/Export on the Dashboard and select New Job.
2. Import Your Plays into a Mirror
Section titled “2. Import Your Plays into a Mirror”A Mirror is a saved copy of your Play history in the Multi-Scrobbler database. Jobs always export from a Mirror.
In the Import section:
- In Import From, select where your Plays come from. For this example, select your Last.fm Source/Client, or upload a file.
- In Mirror, select New mirror and give it a name.
- In Range, keep Everything, or select Between dates to import part of your history.
The Mirror records the date range that each import covers. You can see these ranges on the Mirror’s page (Menu -> Mirrors) after it is created.
3. Export to a Client
Section titled “3. Export to a Client”In the Export section, select the Clients that get the Plays. Each Client exports separately and has its own progress and controls.
Exported Plays go through a Client the same way as live scrobbles:
- The Client applies its own transforms.
- The Client skips Plays that its service already has. See Duplicate Checks.
- The Client sends the Plays to its service one at a time.
The Client always skips Plays that Multi-Scrobbler already sent to it. Because of this, you can export the same Plays two times without duplicates on the service.
You can also set Transforms for the Job. They modify Play data before they go to each Client, the same as a postCompare hook on a Source. You can apply transforms from an existing Source/Client or use a preset. Be aware that exports are only as fast the slowest transform.
Once the Job is created, exporting to a Client occurs depending on the Order you set:
- Newest first - Exporting happens as new Plays are added to the Mirror
- Oldest first - Exporting only starts after the date range is fully imported
4. Start the Job and Watch It
Section titled “4. Start the Job and Watch It”Select Create. The Job page shows the progress of the Import and of each Client’s Export.
You can Pause, Stop, and Resume each part from the Job page.
The Scrobble Interval of a Client sets the export speed. The default is one Play each second. Lower it in the Client’s configuration to export faster.
The duplicate check against the Client’s service also slows large exports. You can turn it off on the Job page while the Export runs. See Duplicate Checks.
Exported Plays do not show in a Client’s Play list by default. To see them, turn on Plays from Export jobs in the list filters.
5. Reuse and Add to a Mirror
Section titled “5. Reuse and Add to a Mirror”Mirror are persisent; they are not deleted after a Job finishes. You can use a Miror again in new Jobs:
- To send the same history to another Client, create a Job, select the Mirror using Existing Mirror, and add the Client in Export.
- To add more history, create a Job that imports into the same Mirror.
A Mirror can hold Plays from different sources. You decide which sources and ranges go together.
For example…
Your Plays from 2008 to 2012 exist only on Last.fm, and a file has your Plays from 2013 onward. Import the file into a Mirror. Then import Last.fm into the same Mirror with a Range of 2008 to 2012. The Mirror exports both as one history.
Only one Job at a time can import into a Mirror.
Plays Already in the Mirror
Section titled “Plays Already in the Mirror”When you import into a Mirror that already has Plays, select what happens where the date ranges overlap:
- Keep (default): Multi-Scrobbler keeps the Plays already in the Mirror and skips new Plays in those dates.
- Replace: Multi-Scrobbler deletes all Plays in the Mirror in the Job’s date range, then imports the new Plays.
6. Clean Up
Section titled “6. Clean Up”Imported Plays, and the records that Clients keep of exported Plays, use disk space. When your exports finish, you can clean them up:
- Compact removes the raw data and transform details of each Play. The Plays stay.
- Delete removes the Plays.
Plays that a Client already sent stay on its service. Cleanup changes only what Multi-Scrobbler keeps in its database. A cleanup cannot start while an import or export that uses the same Plays is running.
You can clean up in three places:
- The Clean Up section of a Job page. It cleans up the Mirror Plays in the Job’s range and the Plays the Job sent to each Client. If you Delete the Job, it asks what to do with the same Plays.
- The Mirror page. You can clean up a date range or delete the whole Mirror. (You should delete the Mirror’s Jobs first.)
- The Job Plays section of a Client page. It cleans up all Plays that Jobs exported to that Client.
Other Jobs can use the same Mirror. A Mirror cleanup affects all of them.
More Details
Section titled “More Details”Import from a File
Section titled “Import from a File”Multi-Scrobbler accepts files from these export tools:
| Exporter | Format | Notes |
|---|---|---|
| lastfm.ghan.nl | JSON or CSV | Recommended. Timestamps to the second, and MusicBrainz IDs are kept. |
| benjaminbenben | CSV | Timestamps only to the minute. See the note below. |
benjaminbenben files
Multi-Scrobbler cannot match minute-only timestamps reliably against other sources. If a benjaminbenben file overlaps other Plays in the Mirror, use Replace for that range, or import it into a Mirror of its own. If you do not, the same Plays can be in the Mirror two times.
You can add a file in two ways:
Upload the file in the Import section of New Job.
Max Upload Size
A reverse proxy in front of Multi-Scrobbler can refuse large uploads. For example, nginx refuses request bodies over 1 MB by default. Raise its limit for Multi-Scrobbler:
client_max_body_size 2g;Or put the file in the imports folder. See the Imports Folder tab.
Put the file in the imports folder of the data directory. Then select File in imports folder in the Job.
- If you did not set
DATA_DIR, useCONFIG_DIR/imports. Make the folder if it does not exist. - If you set
DATA_DIR, useDATA_DIR/imports.
Multi-Scrobbler never deletes files in this folder. Only one Job can use each file.
Rate Limits
Section titled “Rate Limits”The Teal.fm and Rocksky Clients write to your PDS. Your PDS has a rate limit for each account, and a large Export can use all of it. Then your other apps, for example Bluesky, cannot write to your account.
These Clients keep a reserve of points free. When the reserve is reached, the Export waits until the limit resets. You can see the points that are used in the Export section of the Job page.
Failed Plays
Section titled “Failed Plays”If many exported Plays fail in a row, the Export stops and shows the last error. The limit is 10, or the Queue Limit if it is higher. This usually means that the Client configuration is wrong or its service is down. Fix the problem, then select Resume.
Multi-Scrobbler does not retry failed Plays from a Job by itself. To retry them, do one of these:
- On the Client page, turn on Plays from Export jobs in the Plays filters. Set Played Between to the Job’s range. Then select Retry All Unsuccessful in the menu next to Plays.
- Create a new Job with the same Mirror and export to the Client again. The Client sends the failed Plays and skips the Plays it already has.
Export Speed
Section titled “Export Speed”The rate at which Plays are exported from a Job is shown for each Client on the Jobs page. This speed is dependent on several factors, some of which you may need to tune.
Click on Show Metrics in the client Export panel to see a detailed breakdown of processing metrics.
Diagnosing Slow Exports
If an Export is slower than you expect, collect its metrics before you open an issue:
- On the Job page, in the Export section, select Show metrics.
- Select the copy button. It copies the metrics as JSON.
- Add the JSON to your issue.
The metrics show the time that each step of the Export takes, for example reading the Mirror, the duplicate checks, and the requests to the Client’s service. The times cover the last 10 minutes, so copy them while the Export is slow. Select Refresh to get new values.
The same timings are also available from the Prometheus endpoint as multiscrobbler_export_stage_seconds, multiscrobbler_export_wait_seconds, and multiscrobbler_play_stage_seconds.
Transforms
Section titled “Transforms”Multi-Scrobbler exports Plays one at a time. Transforms that call external services (MusicBrainz, Rocksky) wait for each call. If you use Transforms and the metric Process in the Client (shown under Show Metrics) is a large contributor to timing metrics this may indicate Transforms are slowing down your export.
Duplicate Checks
Section titled “Duplicate Checks”Before a Client sends a Play, it compares the Play against two lists:
- The Plays that Multi-Scrobbler already sent to that Client. These are in the Multi-Scrobbler database, so this check is fast.
- The scrobbles on the Client’s service. The Client fetches them from the service, so this check is slow for large exports.
The setting Check downstream services for duplicates controls the second list. It is on by default. If you clear it, the Client uses only the database list, and the Export rate will be much faster.
You can change the setting in two places:
- When you create the Job, in the Advanced settings of the Export section.
- On the Job page, in the Export section. You can change it while the Export runs.
A change applies to the Plays that the Export sends next. Plays that are already in the Client’s queue keep the old setting. The queue holds up to the Job’s Queue Limit of Plays.
Downstream Client Scrobble Processing
Section titled “Downstream Client Scrobble Processing”Some Clients, like Koito, do extra processing when you send a scrobble request. This extra processing can be a significant contributor to how fast, or slow, the scrobble request response is.
If export speed is important for you and Request to the service shown in the Show Metrics toggle is the majority of the processing time then you should attempt to tune your downstream client to reduce scrobble request response time.
Tuning Services
Set these ENVs in the Koito container:
KOITO_DISABLE_MUSICBRAINZ=trueKOITO_DISABLE_COVER_ART_ARCHIVE=trueKOITO_DISABLE_DEEZER=trueKOITO_FETCH_IMAGES_DURING_IMPORT=true
You should be aware that this effectively disables any track/artist/album corrections Koito would normally do when ingesting scrobbles. Only disable these if you are OK with scrobbles being recorded as-is, in Kotio.