Jellyfin extensions

AIOStreams' Jellyfin server speaks the standard Jellyfin API, and its web app works with any Jellyfin server. On top of that it adds a few fields and endpoints, which the web app uses where a server has them and does without where it does not. A server that implements some of them gets those parts of the app; everything else falls back to plain Jellyfin.

Standard Jellyfin clients ignore all of these, so offering them never breaks one.

Finding out what a server offersCopy Anchor Link

GET /System/Info/Public carries an aiostreams object next to the standard fields:

{
  "ServerName": "My server",
  "aiostreams": {
    "logo": "https://…/logo.png",
    "configureUrl": "https://example.com/configure",
    "pinSignIn": false,
    "personal": false,
    "version": { "tag": "v2.35.0", "channel": "stable", "commit": "a1b2c3d4" },
    "features": {
      "users": 1,
      "history": 1,
      "playedUpTo": 1,
      "dropped": 1,
      "refreshVersions": 1,
      "versions": 1,
      "genreRequired": 1,
      "fillers": 1,
      "parties": 1,
      "reports": 1
    },
    "css": null,
    "parties": { "relay": "off", "defaultRelay": null },
    "reports": { "forward": true }
  }
}

The server list reads ServerName and logo before signing in, so a server that hosts several accounts at their own addresses should answer with the one the address names.

The web app checks features for each part it uses, never whether the server is AIOStreams. Each value is that feature's version.

FeaturesCopy Anchor Link

Every /AIOStreams/* endpoint takes the same authorization as the rest of the API. AIOStreams refuses them to API keys, which belong to other tools.

VersionsCopy Anchor Link

A title's versions are its MediaSources. With the versions feature, each carries an aiostreams object:

The version search, quick filters and badges also read these, all optional:

The first version's Id is the item's own id, as in standard Jellyfin, so id is how the app tells versions apart across listings. Without the feature, the next episode settings say nothing of matching versions.

The app loads a title's page with /Items?Ids={id}&Fields=… and no MediaSources field, and asks for versions only when play is pressed. A server that has to run something slow to find versions should only do so for PlaybackInfo, or for a lookup that asks for MediaSources.

In the PlaybackInfo body:

Other clients send neither, so a version that only plays with its headers, or only as a YouTube video, is listed only when the body says the app can play it.

In the query, MediaSourceFields takes a comma-separated list of the MediaSources fields to send, in any case, such as Id,Type,Name,RunTimeTicks,aiostreams. The app asks for those when it lists versions, as it reads nothing else from a list and names a version to play it, which answers with all of it. A list without MediaStreams is also quicker to build, as the server skips the tracks.

Playback reportsCopy Anchor Link

The app adds RunTimeTicks, the length of the file as its player measured it, to the body of POST /Sessions/Playing, /Sessions/Playing/Progress and /Sessions/Playing/Stopped, and of the POST /UserItems/{id}/UserData that saves a position it could not report at the time. AIOStreams judges how much was watched against it, so a title is marked played at the right point even when its runtime from metadata is minutes off the file's. Without it, the server uses the runtime it knows for that version, or the title's.

LibrariesCopy Anchor Link

With genreRequired, a library (a CollectionFolder) whose items only list once a genre is picked carries "aiostreams": { "genreRequired": true }. Asked for without a genre, such a library lists nothing. Its genres come from /Genres?ParentId={library} as usual, and a genre named None stands for all of it. The app opens it on None where that is offered and on its first genre otherwise, and offers All only where None is there to send.

Live TVCopy Anchor Link

The app shows Live TV when the user's views include one with CollectionType livetv, and reads it with Jellyfin's standard Live TV API: /LiveTv/Channels, POST /LiveTv/Programs for a guide, and /Items?SearchTerm=…&IncludeItemTypes=TvChannel for search. A channel plays and reports as any title does.

The view carries "aiostreams": { "guide": true } when one of its catalogs is a guide. The app offers its guide then, or when a channel on the first page of /LiveTv/Channels has a CurrentProgram.

Its groups are the genres of the Live TV view, from /Genres?ParentId={view}. The app asks for a group's channels with /LiveTv/Channels?GenreIds={genre}, which Jellyfin itself doesn't filter by; a server without groups lists no genres there, and the app shows all its channels together. Favourite channels come from /LiveTv/Channels?IsFavorite=true.

A programme its channel keeps carries "aiostreams": { "catchup": true }, and plays from its start by its own id: PlaybackInfo and the stream answer with its archive rather than the channel live. /LiveTv/GuideInfo starts as many days back as the addons' manifests declare catch-up.

TitlesCopy Anchor Link

A title or collection whose addon sends an animatedPoster carries "aiostreams": { "animatedPoster": "…" }, the image's own address. The app plays it over the poster while the card is focused or hovered, and not when the device asks for reduced motion. The poster stays the still that everything else shows.

A title whose addon sends ratings carries them as "aiostreams": { "ratings": [{ "source": "imdb", "value": 8.1, "scale": 10 }, …] }, each on its source's scale, in the addon's order. The app shows a badge for each, in place of the CommunityRating star. That star, and the Rotten Tomatoes CriticRating, are still filled from the list for other clients.

A title no addon in the configuration has details for still opens, named by its id, with an Overview saying so, and carries "aiostreams": { "noMeta": true }. The app shows a page asking for a metadata addon in place of the title's own, with Play kept for a movie.

EpisodesCopy Anchor Link

With fillers, a filler episode carries "aiostreams": { "filler": true } and a recap "aiostreams": { "recap": true }. The web app marks them in episode lists and passes over them when it plays on to the next episode, unless the user turns that off. Without the feature, those settings are hidden.

An episode from /Shows/NextUp carries "aiostreams": { "showWatchedAt": "…" }: when its show was last watched, which Next Up is ordered by, while its own UserData stays the episode's. The web app merges Next Up into Continue watching by it, beside each resume point's LastPlayedDate.

An episode whose Id only its own server resolves, because its series is known by one id and its episodes by another (an anime listed by its MyAnimeList id, with IMDb episode ids), carries "aiostreams": { "sharedId": "…" }: the same episode's id under its own episode id, which any AIOStreams server resolves. A watch party sends it along, so members on other servers open the same episode.

UsersCopy Anchor Link

GET /AIOStreams/Users lists who the signed-in user can switch to:

[
  {
    "user": { "Id": "…", "Name": "Sam" },
    "avatar": "https://…/sam.png",
    "hidden": false,
    "needs": "pin"
  }
]

user is a standard UserDto. needs says what switching to that user asks for: null for nothing, "pin", "password" or "password-pin". A hidden user is listed only to itself, and its own list holds only itself.

POST /AIOStreams/Token with { "UserId": "…", "Pw": "…" } switches: Pw is what needs asked for, with a password and PIN written as password/PIN. It answers like /Users/AuthenticateByName, with 401 for a wrong secret, 404 for an unknown user and 429 after too many tries.

History and activityCopy Anchor Link

GET /AIOStreams/Activity answers the Activity page's header:

{
  "users": [
    {
      "user": { "Id": "…", "Name": "Sam" },
      "avatar": null,
      "hidden": false,
      "locked": true,
      "historyOf": "…",
      "counts": {
        "played": 120,
        "movies": 30,
        "episodes": 90,
        "inProgress": 4,
        "favorites": 12,
        "lastAt": 1790290000000
      }
    }
  ],
  "sessions": []
}

locked is whether the user has a PIN. historyOf is the user whose history this one shares, and counts is null on a user that shares another's. sessions are standard SessionInfoDtos of what is playing now, sent only to the account's own user.

GET /AIOStreams/History?userId=&limit=&cursor=&source= pages through what was watched, newest first. userId narrows it to one user, source=local leaves out plays imported from elsewhere, and cursor is the previous page's:

{
  "items": [
    {
      "userId": "…",
      "itemKey": "…",
      "kind": "episode",
      "played": true,
      "playCount": 1,
      "positionMs": 0,
      "durationMs": 2700000,
      "favorite": false,
      "lastPlayedAt": 1790290000000,
      "sortAt": 1790290000000,
      "origin": "import",
      "tracker": "My tracker",
      "item": { "Id": "…", "Name": "…", "Type": "Episode", "SeriesId": "…" }
    }
  ],
  "cursor": "…"
}

origin is "local" for a play on this server and "import" for one read from a tracker, named in tracker. item is a standard BaseItemDto.

POST /AIOStreams/History/Clear with { "userId": "…", "itemKeys": ["…"] } removes those entries, or the user's whole history without itemKeys, and answers { "cleared": 3 }. GET /AIOStreams/History/Export downloads every visible user's history as one JSON file.

Without history, the app shows the signed-in user's own activity from standard queries: played items sorted by DatePlayed, the TotalRecordCount of played, in-progress and favourite items, and /Sessions.

Played up toCopy Anchor Link

POST /AIOStreams/PlayedUpTo/{episodeId} marks that episode and every earlier one of its series as played, leaving out specials and episodes not aired yet, and answers 204. Without playedUpTo the app does not offer it: marking a long series one episode at a time would take a request each.

DroppedCopy Anchor Link

With dropped, rating a series with the standard Likes=false (POST /UserItems/{id}/Rating) drops it: it leaves Next Up until it is played again, and the drop reaches the user's trackers. The app shows "Drop show" on series only where the server has the feature, since elsewhere the rating would do nothing. It undrops with Likes=true, since clearing the rating would also clear a numeric one.

Watch partiesCopy Anchor Link

With parties, the server offers standard SyncPlay (/SyncPlay/*, /GetUtcTime and the SyncPlayCommand and SyncPlayGroupUpdate socket messages) to Jellyfin apps, and runs a relay that the web app's watch parties use, on the same server and across servers:

aiostreams.parties.relay is off (only the server's users, who send their token), users (its users start parties, anyone with an invite joins) or public. While it is off, defaultRelay is the address of a relay the server suggests for parties with people elsewhere, or null. With no parties object at all, the web app suggests https://aiostreams.viren070.me.

An invite is a relay and a 10-character code from 0123456789ABCDEFGHJKMNPQRSTVWXYZ. PBKDF2 with SHA-256 over the code, salt aiostreams-party-1 and 100,000 iterations gives 48 bytes: the first 16 are the party's id on the relay, in base64url, and the other 32 an AES-GCM key. Names, chat, reactions and what plays travel sealed with that key, as base64url of a 12-byte nonce followed by the ciphertext of their JSON, so a relay keeps only timing. The messages themselves are typed in packages/core/src/sync/protocol.ts.

PlaylistsCopy Anchor Link

The web app keeps playlists with Jellyfin's standard /Playlists API, and lists them with /Items?IncludeItemTypes=Playlist&Recursive=true.

Before adding, the app reads the playlist's ItemIds and leaves out what is already there, as Jellyfin keeps a title twice. It removes and moves entries by their PlaylistItemId.

A title played from a playlist carries the playlist's id, so it plays on to the playlist's next title rather than its next episode, as do the player's previous and next buttons. The app hands the rest of the playlist to a player with a queue of its own, and to a watch party.

Remote control and linksCopy Anchor Link

Nothing here is an extension: the web app takes Jellyfin's standard remote control. Each time its session socket opens, it posts /Sessions/Capabilities/Full with Video and the commands DisplayContent, SetAudioStreamIndex and SetSubtitleStreamIndex, and media control. On the socket it acts on:

Its progress reports carry AudioStreamIndex and SubtitleStreamIndex in the same terms. Play with PlayNext or PlayLast, while it plays a title from a playlist, adds the titles to that playlist, after the one playing or at the end; otherwise they are ignored.

The desktop and Android apps also open links, so a page elsewhere can open a title in them:

server is the address the app signs in to, which the item ids belong to. With another one, the app asks before switching to it, or offers to add one it doesn't know; without server, it opens the title on the server it is on. season, episode, start and version are optional, and every value is URL-encoded.

ReportsCopy Anchor Link

With reports, POST /AIOStreams/Reports keeps a report for the server's admins, who read it in the dashboard under Reports:

{
  "kind": "crash",
  "platform": "android",
  "appVersion": "0.5.1",
  "device": "Sony BRAVIA",
  "note": "It closed as the film started",
  "summary": "Native crash: SIGSEGV …",
  "details": "…the whole report and the recent log…",
  "forward": true
}

kind is crash or report, summary a few lines a list shows the first of, and details the rest, kept to 192 KB by cutting the middle. The answer is the code to read the report by: { "code": "ABC-DEFG", "forwarded": "HJK-MNPQ", "forwardError": null }. With forward, and aiostreams.reports.forward true, the server also passes it on to the developer's server (APP_REPORTS_FORWARD_TO) and gives its code as forwarded, or why it couldn't as forwardError. Passwords, tokens and keys are taken out of what is kept, and configuration ids out of what is passed on. A server whose APP_REPORTS is public takes passed-on reports at POST /api/v1/reports.

What stays with AIOStreamsCopy Anchor Link

These belong to how AIOStreams signs in to a configuration, and other servers have no use for them:

• Signing in with a configuration's UUID or alias, <uuid>/<user> as the user name and password/PIN as the password, and the configSignIn feature that announces it.
• Signing in with a PIN alone (pinSignIn), on the addresses that name a configuration: /jellyfin/<uuid>/<encrypted password>, /jellyfin/u/<alias> and /jellyfin/p/<code>.
• A separate user's own address, /jellyfin/p/<code> (personal), which lists and signs in that one user and no other.
• The web app signing in from the configure page, through AIOStreams' own API.