Resource API Refactor

Just like with the Quicklists application, we are now going to make some modifications to this application to utilise resource and linkedSignal.

This refactor will be a lot more advanced, and also perhaps a bit more open to interpretation as to whether it’s actually an improvement or not.

This application has made heavy use of RxJS. It’s an application that has a lot of asynchronous stuff happening, even streaming results from the Reddit API and having them appear as each batch comes in, and this is a task that RxJS is typically quite suited for.

In fact, when I first attempted to refactor this application I did it mostly for experimentation purposes and seeing how far I could push resource and linkedSignal. I didn’t expect the result to be actually good.

But it was (at least in my opinion). Again, I don’t think there is a clear cut winner here, but on the balance I prefer the refactored approach that we are about to implement.

Refactoring the RedditService

I think the best way to approach this will be to just show the completely refactored service from the start, and then we will talk through how each bit works.

The good thing is that conceptually the flow of data and how things work is basically the same as what we have already discussed with RxJS, it’s just the mechanisms we are achieving to use it are a bit different.

import { Injectable, inject, linkedSignal } from '@angular/core';
import { rxResource, toSignal } from '@angular/core/rxjs-interop';
import { HttpClient } from '@angular/common/http';
import { Gif, RedditPost, RedditResponse } from '../interfaces';
import { FormControl } from '@angular/forms';
import { EMPTY } from 'rxjs';
import {
  reduce,
  debounceTime,
  distinctUntilChanged,
  expand,
  map,
  startWith,
} from 'rxjs/operators';

@Injectable({ providedIn: 'root' })
export class RedditService {
  private http = inject(HttpClient);
  private gifsPerPage = 5;

  subredditFormControl = new FormControl();

  //sources
  private subredditChanged$ = this.subredditFormControl.valueChanges.pipe(
    debounceTime(300),
    distinctUntilChanged(),
    startWith('gifs'),
    map((subreddit) => (subreddit.length ? subreddit : 'gifs')),
  );
  subreddit = toSignal(this.subredditChanged$);

  paginateAfter = linkedSignal({
    source: this.subreddit,
    computation: () => null as string | null,
  });

  gifsLoaded = rxResource({
    params: () => ({
      subreddit: this.subreddit(),
      paginateAfter: this.paginateAfter(),
    }),
    stream: ({ params }) =>
      this.fetchRecursivelyFromReddit(params.subreddit, params.paginateAfter),
  });

  gifs = linkedSignal<ReturnType<typeof this.gifsLoaded.value>, Gif[]>({
    source: this.gifsLoaded.value,
    computation: (source, prev) => {
      // initial and page loads
      if (typeof source === 'undefined') return prev?.value ?? [];

      // clear on subreddit change
      if (
        !prev ||
        !prev.value[0]?.permalink.startsWith(`/r/${source.subreddit}`)
      )
        return source.gifs;

      // accumulate values on paginate
      return [...prev.value, ...source.gifs];
    },
  });

  private fetchFromReddit(
    subreddit: string,
    after: string | null,
    gifsRequired: number,
  ) {
    return this.http
      .get<RedditResponse>(
        `https://www.reddit.com/r/${subreddit}/hot/.json?limit=100` +
          (after ? `&after=${after}` : ''),
      )
      .pipe(
        map((response) => {
          const posts = response.data.children;
          let gifs = this.convertRedditPostsToGifs(posts);
          let paginateAfter = posts.length
            ? posts[posts.length - 1].data.name
            : null;

          return {
            gifs,
            gifsRequired,
            paginateAfter,
            subreddit,
          };
        }),
      );
  }

  private fetchRecursivelyFromReddit(
    subreddit: string,
    paginateAfter: string | null,
  ) {
    return this.fetchFromReddit(
      subreddit,
      paginateAfter,
      this.gifsPerPage,
    ).pipe(
      // A single request might not give us enough valid gifs for a
      // full page, as not every post is a valid gif
      // Keep fetching more data until we do have enough for a page
      expand((response, index) => {
        const { gifs, gifsRequired, paginateAfter } = response;
        const remainingGifsToFetch = gifsRequired - gifs.length;
        const maxAttempts = 5;

        const shouldKeepTrying =
          remainingGifsToFetch > 0 &&
          index < maxAttempts &&
          paginateAfter !== null;

        return shouldKeepTrying
          ? this.fetchFromReddit(subreddit, paginateAfter, remainingGifsToFetch)
          : EMPTY;
      }),
      map((response) => {
        const { gifs, gifsRequired } = response;
        const remainingGifsToFetch = gifsRequired - gifs.length;

        if (remainingGifsToFetch < 0) {
          // trim to page size
          const trimmedGifs = response.gifs.slice(0, remainingGifsToFetch);
          return {
            ...response,
            gifs: trimmedGifs,
            paginateAfter: trimmedGifs[trimmedGifs.length - 1].name,
          };
        }

        return response;
      }),
      reduce(
        (acc, curr) => ({
          ...curr,
          gifs: [...acc.gifs, ...curr.gifs],
        }),
        {
          gifs: [] as Gif[],
          paginateAfter: null as string | null,
          gifsRequired: this.gifsPerPage,
          subreddit: 'gifs',
        },
      ),
    );
  }

  private convertRedditPostsToGifs(posts: RedditPost[]) {
    const defaultThumbnails = ['default', 'none', 'nsfw'];

    return posts
      .map((post) => {
        const thumbnail = post.data.thumbnail;
        const modifiedThumbnail = defaultThumbnails.includes(thumbnail)
          ? `/assets/${thumbnail}.png`
          : thumbnail;

        const validThumbnail =
          modifiedThumbnail.endsWith('.jpg') ||
          modifiedThumbnail.endsWith('.png');

        return {
          src: this.getBestSrcForGif(post),
          author: post.data.author,
          name: post.data.name,
          permalink: post.data.permalink,
          title: post.data.title,
          thumbnail: validThumbnail ? modifiedThumbnail : `/assets/default.png`,
          comments: post.data.num_comments,
        };
      })
      .filter((post): post is Gif => post.src !== null);
  }

  private getBestSrcForGif(post: RedditPost) {
    // If the source is in .mp4 format, leave unchanged
    if (post.data.url.indexOf('.mp4') > -1) {
      return post.data.url;
    }

    // If the source is in .gifv or .webm formats, convert to .mp4 and return
    if (post.data.url.indexOf('.gifv') > -1) {
      return post.data.url.replace('.gifv', '.mp4');
    }

    if (post.data.url.indexOf('.webm') > -1) {
      return post.data.url.replace('.webm', '.mp4');
    }

    // If the URL is not .gifv or .webm, check if media or secure media is available
    if (post.data.secure_media?.reddit_video) {
      return post.data.secure_media.reddit_video.fallback_url;
    }

    if (post.data.media?.reddit_video) {
      return post.data.media.reddit_video.fallback_url;
    }

    // If media objects are not available, check if a preview is available
    if (post.data.preview?.reddit_video_preview) {
      return post.data.preview.reddit_video_preview.fallback_url;
    }

    // No useable formats available
    return null;
  }
}

Let’s first look at our first few sources:

  //sources
  private subredditChanged$ = this.subredditFormControl.valueChanges.pipe(
    debounceTime(300),
    distinctUntilChanged(),
    startWith('gifs'),
    map((subreddit) => (subreddit.length ? subreddit : 'gifs')),
  );
  subreddit = toSignal(this.subredditChanged$);

  paginateAfter = linkedSignal({
    source: this.subreddit,
    computation: () => null as string | null,
  });

  gifsLoaded = rxResource({
    params: () => ({
      subreddit: this.subreddit(),
      paginateAfter: this.paginateAfter(),
    }),
    stream: ({ params }) =>
      this.fetchRecursivelyFromReddit(params.subreddit, params.paginateAfter),
  });

The subredditChanged source is still mostly the same, it still uses the same RxJS operators, we are just converting it into a signal.

The introduction of this paginateAfter is where we first start to see significant changes. This is a signal that serves two purposes: it keeps track of our “last gif seen” so we know where to paginate from, and it is also how we trigger pagination. If we want to trigger loading more data we would .set this linkedSignal with the name of our “last gif”.

But, we also want this paginateAfter to have its value reset when the subreddit changes. So, we have this.subreddit as a source for the linkedSignal which will cause the computation to run every time it changes. This resets it back to null. But we can still update its value from null to the name of our “last gif” from elsewhere.

Then we get to the gifsLoaded source, which is what handles loading our data from the Reddit API:

  gifsLoaded = rxResource({
    params: () => ({
      subreddit: this.subreddit(),
      paginateAfter: this.paginateAfter(),
    }),
    stream: ({ params }) =>
      this.fetchRecursivelyFromReddit(params.subreddit, params.paginateAfter),
  });

We want this loading of gifs to be triggered under two conditions:

When the subreddit changes
When the paginateAfter value changes

So, we supply both of those in our params function. This will then trigger the stream using those values.

But, notice that we are actually using rxResource here. We will still be using RxJS to handle fetching the gifs recursively from the Reddit API by using the expand operator as before.

An important change here is that we have included this reduce operator after the expand:

      reduce(
        (acc, curr) => ({
          ...curr,
          gifs: [...acc.gifs, ...curr.gifs],
        }),
        {
          gifs: [] as Gif[],
          paginateAfter: null as string | null,
          gifsRequired: this.gifsPerPage,
          subreddit: 'gifs',
        },
      ),

An important thing to keep in mind when using rxResource is that, even though we can supply an observable as the loader, it will only return the first emission from the observable we supply to it. With our expand set up, it will cause the stream to emit the data it retrieves each time the stream “expands”. But since we are using rxResource it means we would only get the first set of results returned.

By using reduce we are able to collect all of the stream emissions into one single emission. This has the downside of requiring us to wait until we have all of the valid gifs before we are able to see anything, whereas with the more RxJS centric approach we are able to “stream” results in as they come.

This brings us to the final piece of this refactoring puzzle:

  gifs = linkedSignal<ReturnType<typeof this.gifsLoaded.value>, Gif[]>({
    source: this.gifsLoaded.value,
    computation: (source, prev) => {
      // initial and page loads
      if (typeof source === 'undefined') return prev?.value ?? [];

      // clear on subreddit change
      if (
        !prev ||
        !prev.value[0]?.permalink.startsWith(`/r/${source.subreddit}`)
      )
        return source.gifs;

      // accumulate values on paginate
      return [...prev.value, ...source.gifs];
    },
  });

This is perhaps the most unintuitive and advanced of the changes we have made. First, we are supplying this type manually to linkedSignal:

linkedSignal<ReturnType<typeof this.gifsLoaded.value>, Gif[]>

This is basically saying that our source for the linkedSignal is going to be the values from our gifsLoaded resource (the data we just got from the Reddit API), and that this computation is going to return an array of gifs (Gif[]).

We want this gifs linkedSignal to be the gifs we intend to consume in the application.

But this is a complicated scenario because we can’t just take the values we get from gifsLoaded and set them into our gifs signal. We are using infinite scrolling, and we want newly loaded gifs added to the existing gifs…

That is unless we have changed subreddits, in that case we actually do want to clear out the existing gifs and only return the new ones.

That is what this computation is handling:

    computation: (source, prev) => {
      // initial and page loads
      if (typeof source === 'undefined') return prev?.value ?? [];

      // clear on subreddit change
      if (
        !prev ||
        !prev.value[0]?.permalink.startsWith(`/r/${source.subreddit}`)
      )
        return source.gifs;

      // accumulate values on paginate
      return [...prev.value, ...source.gifs];
    },

As well as passing in the value from our current source (i.e. the value from gifsLoaded), we can also pass in prev which is the result from the computation the last time it ran (i.e. our current array of gifs).

The basic scenario here is the last one, just combine the new gifs with the current gifs:

return [...prev.value, ...source.gifs];

But our source will be undefined initially and during page loads. When that is happening we make sure to just return the current gifs if there are any, otherwise we return an empty array:

if (typeof source === 'undefined') return prev?.value ?? [];

And if the results that have just come in are from a different subreddit than the previous results, we know we want to clear out all the gifs from the old subreddit and only return the new ones:

  if (
    !prev ||
    !prev.value[0]?.permalink.startsWith(`/r/${source.subreddit}`)
  )
    return source.gifs;

…And that’s about it for the changes. Of course, just as we did with the Quicklists application we will also need to update how this service is consumed in the application, e.g. we will need to trigger page loads like this now:

    <app-gif-list
      [gifs]="redditService.gifs()"
      infiniteScroll
      (scrolled)="
        redditService.paginateAfter.set(
          redditService.gifsLoaded.value()?.paginateAfter ?? null
        )
      "
      class="grid-container"
    />

    @if (redditService.gifsLoaded.isLoading()) {
      <mat-progress-spinner mode="indeterminate" diameter="50" />
    }

Notice that I have also updated the spinner to show based on the isLoading signal of the gifsLoaded resource.

Summary

The usage of rxResource and linkedSignal here is much more advanced than what we used previously in the Quicklists application. This is actually probably more advanced than I think most situations you are likely to run into.

Whether this approach is better or worse than the original implementation is up for debate. Personally, I think it is a bit better.