This tutorial explains how Video Hub runs the automatic imports behind the scenes: when an import runs, which importer it uses, how much of a feed it reads, and what happens when it reaches the end of the feed. Knowing this helps you choose an import pace that keeps your site up to date without overloading your server or using too much of your YouTube API quota. To create an importer, see Automatic video import.
How an import is started
The imports run in the background, through Action Scheduler, the job queue used by WooCommerce and many other plugins, which Video Hub includes. Video Hub schedules a single recurring job that runs at the frequency you set. Your visitors never wait for an import, and two imports never run at the same time.
Action Scheduler is started by WP-Cron, which WordPress runs when the site has visitors. This works on most sites. On a site with little traffic, or when the host disables WP-Cron, the imports fall behind; the Import status box of the Automatic import page then shows that the imports are stalled. The fix is a server cron job, explained in Troubleshooting automatic imports.
The import pace
In Videos → Settings, tab Import options, the Import pace sets two values:
- how many videos an import run reads from YouTube, from 1 to 50;
- how often an import runs, from every minute to once a day.
Importing a video is more work than it looks. For each video, WordPress inserts the post, imports the featured image into the Media Library when that option is on, and assigns the categories and the tags. Every run also uses YouTube API quota. Start with 10–20 videos every 5 minutes, which works on most websites, and adjust from there.
The import queue
Every run imports from a single importer. The importers in the queue take turns, in the order they were created: the first run imports from the first importer, the next run from the second one, and so on, then the queue starts over.
An importer with Repeat import in queue set to 2 or more gets that many runs in a row before the next importer. A full round of the queue takes as many runs as the sum of these values. For example, with 3 importers, one of them set to repeat 2 times, a round takes 4 runs; with an import every 5 minutes, that’s 20 minutes. The Import status box shows this as Queue round.
An importer that returns a YouTube error waits before it is tried again: 5 minutes after the first error, 10 after the second, and so on, while the other importers keep their turns. With the YouTube errors option of the settings, an importer that returns 5 errors in a row is taken out of the queue.
Reading a feed, one page at a time
A run reads one page of the feed, as many videos as the import pace allows, and imports the videos that are not on your site yet. The next turn of the importer continues with the next page. A channel with 2,000 videos, read 20 at a time, needs 100 turns to be read completely.
Some of the videos read are not imported. They are counted as skipped:
- videos already imported, unless the settings allow duplicates;
- videos you deleted from the website, when the importer has Don’t import deleted videos again checked;
- YouTube Shorts, when the importer skips them;
- videos outside the duration limits of the importer.
Private videos, and unlisted videos when the settings don’t allow them, are not imported either. All of them still count as read, so they don’t make the import take longer.
The time estimate
Under the progress bar, the list of importers shows how long each importer needs to read the rest of its feed. The estimate is:
videos left to read ÷ videos per run ÷ repeats in queue × queue round
For example, an importer with 1,544 videos left, reading 20 videos per run, in a queue round of 2 minutes, needs 78 runs, about 2 hours and 36 minutes, shown as “About 3 hours left”. When the importer has an Import if published after date, the estimate is a maximum, because the import stops at that date. A paused importer shows the time it would need once started. Read more in Import estimate and activity log.
When the end of the feed is reached
What happens next depends on When finished, import only new videos:
- checked: the next runs check only the beginning of the feed and import the new videos. The new videos are found within one queue round. This is the best choice for most channels.
- not checked: the importer reads the whole feed again from the beginning. The videos already imported are skipped, and the ones that are missing, for example the videos you deleted, are imported again. New videos are found only when the importer gets back to the beginning of the feed.
New videos while a large feed is imported
Reading a large feed can take hours or days, and the videos published on YouTube in the meantime are found only after the importer reaches the end of the feed. Import new videos right away solves this: a second process, at the same frequency as the imports, checks the beginning of the feed and imports the new videos, while the importer keeps reading the older ones. When several importers use the option, they take turns too. Each check uses YouTube API quota, so check the option only for the feeds that need it.
Reset, pause and start
- Pause takes an importer out of the queue and Start adds it back. It continues from where it stopped.
- Reset makes the importer read the feed again from its first video and clears its errors. The videos already imported are skipped. Its activity log and totals are kept, and the reset is recorded in the log.
Conclusion
Keep the number of videos per run realistic for your server, give the importers that matter most more runs with Repeat import in queue, and check When finished, import only new videos for channels. Follow the estimate and the Activity page of each importer to see that everything runs as expected.
See also
- Automatic video import — create and manage the importers.
- Import estimate and activity log — follow the progress of an importer.
- Troubleshooting automatic imports — stalled imports, errors and missing videos.
- Video import options — all the import settings.