label: Example URLs
description: |
Provide all kinds of example URLs for which support should be added
- value: |
+ placeholder: |
- Single video: https://www.youtube.com/watch?v=BaW_jenozKc
- Single video: https://youtu.be/BaW_jenozKc
- Playlist: https://www.youtube.com/playlist?list=PL4lCao7KL_QFVb7Iudeipvc2BCavECqzc
contact_links:
- name: Get help from the community on Discord
url: https://discord.gg/H5MNcFW63r
- about: Join the yt-dlp Discord for community-powered support!
\ No newline at end of file
+ about: Join the yt-dlp Discord for community-powered support!
name: Broken site support
description: Report broken or misfunctioning site
-labels: [triage, extractor-bug]
+labels: [triage, site-bug]
body:
- type: checkboxes
id: checklist
name: Site feature request
-description: Request a new functionality for a site
+description: Request a new functionality for a supported site
labels: [triage, site-enhancement]
body:
- type: checkboxes
placeholder: WRITE DESCRIPTION HERE
validations:
required: true
+ - type: textarea
+ id: log
+ attributes:
+ label: Verbose log
+ description: |
+ Provide the complete verbose output of yt-dlp that demonstrates the need for the enhancement.
+ Add the `-Uv` flag to your command line you run yt-dlp with (`yt-dlp -Uv <your command line>`), copy the WHOLE output and insert it below.
+ It should look similar to this:
+ placeholder: |
+ [debug] Command-line config: ['-Uv', 'http://www.youtube.com/watch?v=BaW_jenozKc']
+ [debug] Portable config file: yt-dlp.conf
+ [debug] Portable config: ['-i']
+ [debug] Encodings: locale cp1252, fs utf-8, stdout utf-8, stderr utf-8, pref cp1252
+ [debug] yt-dlp version %(version)s (exe)
+ [debug] Python version 3.8.8 (CPython 64bit) - Windows-10-10.0.19041-SP0
+ [debug] exe versions: ffmpeg 3.0.1, ffprobe 3.0.1
+ [debug] Optional libraries: Cryptodome, keyring, mutagen, sqlite, websockets
+ [debug] Proxy map: {}
+ yt-dlp is up to date (%(version)s)
+ <more lines>
+ render: shell
+ validations:
+ required: true
name: Bug report
description: Report a bug unrelated to any particular site or extractor
-labels: [triage,bug]
+labels: [triage, bug]
body:
- type: checkboxes
id: checklist
-name: Feature request request
+name: Feature request
description: Request a new functionality unrelated to any particular site or extractor
labels: [triage, enhancement]
body:
description: |
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
options:
- - label: I'm asking a question and not reporting a bug/feature request
+ - label: I'm asking a question and **not** reporting a bug/feature request
required: true
- label: I've looked through the [README](https://github.com/yt-dlp/yt-dlp#readme)
required: true
description: |
Ask your question in an arbitrary form.
Please make sure it's worded well enough to be understood, see [is-the-description-of-the-issue-itself-sufficient](https://github.com/ytdl-org/youtube-dl#is-the-description-of-the-issue-itself-sufficient).
- Provide any additional information and as much context and examples as possible
+ Provide any additional information and as much context and examples as possible.
+ If your question contains "isn't working" or "can you add", this is most likely the wrong template
placeholder: WRITE QUESTION HERE
validations:
required: true
# Config
*.conf
-*.spec
cookies
*cookies.txt
.netrc
# Downloaded
-*.3gp
*.annotations.xml
-*.ape
*.aria2
-*.avi
*.description
-*.desktop
*.dump
-*.flac
-*.flv
*.frag
+*.frag.aria2
*.frag.urls
*.info.json
+*.live_chat.json
+*.part*
+*.unknown_video
+*.ytdl
+.cache/
+
+*.3gp
+*.ape
+*.avi
+*.desktop
+*.flac
+*.flv
*.jpeg
*.jpg
-*.live_chat.json
*.m4a
*.m4v
*.mhtml
*.mp4
*.ogg
*.opus
-*.part
-*.part-*
*.png
*.sbv
*.srt
*.swf
*.swp
*.ttml
-*.unknown_video
*.url
*.vtt
*.wav
*.webloc
*.webm
*.webp
-*.ytdl
-.cache/
# Allow config/media files in testdata
!test/**
*.1
*.bash-completion
*.fish
-*.exe
*.tar.gz
*.zsh
*.spec
In any case, thank you very much for your contributions!
+**Tip:** To test extractors that require login information, create a file `test/local_parameters.json` and add `"usenetrc": true` or your username and password in it:
+```json
+{
+ "username": "your user name",
+ "password": "your password"
+}
+```
## yt-dlp coding conventions
[](https://github.com/sponsors/coletdjnz)
* YouTube improvements including: age-gate bypass, private playlists, multiple-clients (to avoid throttling) and a lot of under-the-hood improvements
+* Added support for downloading YoutubeWebArchive videos
.PHONY: all clean install test tar pypi-files completions ot offlinetest codetest supportedsites
clean-test:
- rm -rf *.3gp *.annotations.xml *.ape *.avi *.description *.dump *.flac *.flv *.frag *.frag.aria2 *.frag.urls \
- *.info.json *.jpeg *.jpg *.live_chat.json *.m4a *.m4v *.mkv *.mp3 *.mp4 *.ogg *.opus *.part* *.png *.sbv *.srt \
- *.swf *.swp *.ttml *.vtt *.wav *.webm *.webp *.mhtml *.mov *.unknown_video *.desktop *.url *.webloc *.ytdl \
- test/testdata/player-*.js tmp/
+ rm -rf test/testdata/player-*.js tmp/ *.annotations.xml *.aria2 *.description *.dump *.frag \
+ *.frag.aria2 *.frag.urls *.info.json *.live_chat.json *.part* *.unknown_video *.ytdl \
+ *.3gp *.ape *.avi *.desktop *.flac *.flv *.jpeg *.jpg *.m4a *.m4v *.mhtml *.mkv *.mov *.mp3 \
+ *.mp4 *.ogg *.opus *.png *.sbv *.srt *.swf *.swp *.ttml *.url *.vtt *.wav *.webloc *.webm *.webp
clean-dist:
rm -rf yt-dlp.1.temp.md yt-dlp.1 README.txt MANIFEST build/ dist/ .coverage cover/ yt-dlp.tar.gz completions/ \
yt_dlp/extractor/lazy_extractors.py *.spec CONTRIBUTING.md.tmp yt-dlp yt-dlp.exe yt_dlp.egg-info/ AUTHORS .mailmap
# NEW FEATURES
-* Based on **youtube-dl 2021.06.06 [commit/379f52a](https://github.com/ytdl-org/youtube-dl/commit/379f52a4954013767219d25099cce9e0f9401961)** and **youtube-dlc 2020.11.11-3 [commit/98e248f](https://github.com/blackjack4494/yt-dlc/commit/98e248faa49e69d795abc60f7cdefcf91e2612aa)**: You get all the features and patches of [youtube-dlc](https://github.com/blackjack4494/yt-dlc) in addition to the latest [youtube-dl](https://github.com/ytdl-org/youtube-dl)
+* Based on **youtube-dl 2021.12.17 [commit/5014bd6](https://github.com/ytdl-org/youtube-dl/commit/5014bd67c22b421207b2650d4dc874b95b36dda1)** and **youtube-dlc 2020.11.11-3 [commit/f9401f2](https://github.com/blackjack4494/yt-dlc/commit/f9401f2a91987068139c5f757b12fc711d4c0cee)**: You get all the features and patches of [youtube-dlc](https://github.com/blackjack4494/yt-dlc) in addition to the latest [youtube-dl](https://github.com/ytdl-org/youtube-dl)
* **[SponsorBlock Integration](#sponsorblock-options)**: You can mark/remove sponsor sections in youtube videos by utilizing the [SponsorBlock](https://sponsor.ajay.app) API
Some of yt-dlp's default options are different from that of youtube-dl and youtube-dlc:
* The options `--auto-number` (`-A`), `--title` (`-t`) and `--literal` (`-l`), no longer work. See [removed options](#Removed) for details
-* `avconv` is not supported as as an alternative to `ffmpeg`
+* `avconv` is not supported as an alternative to `ffmpeg`
* The default [output template](#output-template) is `%(title)s [%(id)s].%(ext)s`. There is no real reason for this change. This was changed before yt-dlp was ever made public and now there are no plans to change it back to `%(title)s-%(id)s.%(ext)s`. Instead, you may use `--compat-options filename`
* The default [format sorting](#sorting-formats) is different from youtube-dl and prefers higher resolution and better codecs rather than higher bitrates. You can use the `--format-sort` option to change this to any order you prefer, or use `--compat-options format-sort` to use youtube-dl's sorting order
* The default format selector is `bv*+ba/b`. This means that if a combined video + audio format that is better than the best video-only format is found, the former will be preferred. Use `-f bv+ba/b` or `--compat-options format-spec` to revert this
```
```
-sudo aria2c https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp -o /usr/local/bin/yt-dlp
+sudo aria2c https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp --dir /usr/local/bin -o yt-dlp
sudo chmod a+rx /usr/local/bin/yt-dlp
```
While all the other dependencies are optional, `ffmpeg` and `ffprobe` are highly recommended
-* [**ffmpeg** and **ffprobe**](https://www.ffmpeg.org) - Required for [merging separate video and audio files](#format-selection) as well as for various [post-processing](#post-processing-options) tasks. Licence [depends on the build](https://www.ffmpeg.org/legal.html)
+* [**ffmpeg** and **ffprobe**](https://www.ffmpeg.org) - Required for [merging separate video and audio files](#format-selection) as well as for various [post-processing](#post-processing-options) tasks. License [depends on the build](https://www.ffmpeg.org/legal.html)
* [**mutagen**](https://github.com/quodlibet/mutagen) - For embedding thumbnail in certain formats. Licensed under [GPLv2+](https://github.com/quodlibet/mutagen/blob/master/COPYING)
* [**pycryptodomex**](https://github.com/Legrandin/pycryptodome) - For decrypting AES-128 HLS streams and various other data. Licensed under [BSD2](https://github.com/Legrandin/pycryptodome/blob/master/LICENSE.rst)
* [**websockets**](https://github.com/aaugustin/websockets) - For downloading over websocket. Licensed under [BSD3](https://github.com/aaugustin/websockets/blob/main/LICENSE)
except Exception:
GIT_HEAD = None
-VERSION_FILE = f'''
+VERSION_FILE = f'''\
# Autogenerated by devscripts/update-version.py
__version__ = {VERSION!r}
RELEASE_GIT_HEAD = {GIT_HEAD!r}
-'''.lstrip()
+'''
with open('yt_dlp/version.py', 'wt') as f:
f.write(VERSION_FILE)
--- /dev/null
+---
+orphan: true
+---
+```{include} ../Contributing.md
+```
test('%(title5)+#U', 'a\u0301e\u0301i\u0301 A')
test('%(height)D', '1K')
test('%(height)5.2D', ' 1.08K')
- test('%(title4).10F', ('foo \'bar\' ', 'foo \'bar\'#'))
test('%(title4)#F', 'foo_bar_test')
+ test('%(title4).10F', ('foo \'bar\' ', 'foo \'bar\'' + ('#' if compat_os_name == 'nt' else ' ')))
if compat_os_name == 'nt':
test('%(title4)q', ('"foo \\"bar\\" test"', "'foo _'bar_' test'"))
test('%(formats.:.id)#q', ('"id 1" "id 2" "id 3"', "'id 1' 'id 2' 'id 3'"))
'https://www.youtube.com/s/player/f1ca6900/player_ias.vflset/en_US/base.js',
'cu3wyu6LQn2hse', 'jvxetvmlI9AN9Q',
),
+ (
+ 'https://www.youtube.com/s/player/8040e515/player_ias.vflset/en_US/base.js',
+ 'wvOFaY-yjgDuIEg5', 'HkfBFDHmgw4rsw',
+ ),
]
self.write_debug('Additional URLs: "%s"' % '", "'.join(additional_urls))
ie_result['additional_entries'] = [
self.extract_info(
- url, download, extra_info,
+ url, download, extra_info=extra_info,
force_generic_extractor=self.params.get('force_generic_extractor'))
for url in additional_urls
]
info_dict['id'], automatic_captions, 'automatic captions')
self.list_subtitles(info_dict['id'], subtitles, 'subtitles')
if self.params.get('listformats') or interactive_format_selection:
- if not info_dict.get('formats') and not info_dict.get('url'):
- self.to_screen('%s has no formats' % info_dict['id'])
- else:
- self.list_formats(info_dict)
+ self.list_formats(info_dict)
if list_only:
# Without this printing, -F --print-json will not work
self.__forced_printings(info_dict, self.prepare_filename(info_dict), incomplete=True)
return headers
def list_formats(self, info_dict):
+ if not info_dict.get('formats') and not info_dict.get('url'):
+ self.to_screen('%s has no formats' % info_dict['id'])
+ return
+ self.to_screen('[info] Available formats for %s:' % info_dict['id'])
+
formats = info_dict.get('formats', [info_dict])
new_format = self.params.get('listformats_table', True) is not False
if new_format:
delim,
format_field(f, 'filesize', ' \t%s', func=format_bytes) + format_field(f, 'filesize_approx', '~\t%s', func=format_bytes),
format_field(f, 'tbr', '\t%dk'),
- shorten_protocol_name(f.get('protocol', '').replace('native', 'n')),
+ shorten_protocol_name(f.get('protocol', '')),
delim,
format_field(f, 'vcodec', default='unknown').replace(
'none',
if f.get('preference') is None or f['preference'] >= -1000]
header_line = ['format code', 'extension', 'resolution', 'note']
- self.to_screen(
- '[info] Available formats for %s:' % info_dict['id'])
self.to_stdout(render_table(
header_line, table,
extra_gap=(0 if new_format else 1),
)
from .compat import (
compat_getpass,
+ compat_os_name,
compat_shlex_quote,
workaround_optparse_bug9161,
)
if opts.batchfile is not None:
try:
if opts.batchfile == '-':
- write_string('Reading URLs from stdin:\n')
+ write_string('Reading URLs from stdin - EOF (%s) to end:\n' % (
+ 'Ctrl+Z' if compat_os_name == 'nt' else 'Ctrl+D'))
batchfd = sys.stdin
else:
batchfd = io.open(
if len(dur) == 2 and all(t is not None for t in dur):
remove_ranges.append(tuple(dur))
continue
- parser.error(f'invalid --remove-chapters time range {regex!r}. Must be of the form ?start-end')
+ parser.error(f'invalid --remove-chapters time range {regex!r}. Must be of the form *start-end')
try:
remove_chapters_patterns.append(re.compile(regex))
except re.error as err:
'status': 'finished',
'total_bytes': os.path.getsize(encodeFilename(filename)),
}, info_dict)
+ self._finish_multiline_status()
return True, False
if subtitle is False:
yield {
**base_format,
'format_id': join_nonempty('sec', height),
- 'url': re.sub(r'(QualityLevels\()\d+(\))', fr'\<1>{bitrate}\2', base_url),
+ 'url': re.sub(r'(QualityLevels\()\d+(\))', fr'\1{bitrate}\2', base_url),
'width': int_or_none(video_quality.attrib.get('MaxWidth')),
'tbr': bitrate / 1000.0,
'height': height,
kwargs = {
'video_id': e.video_id or self.get_temp_id(url),
'ie': self.IE_NAME,
- 'tb': e.traceback,
+ 'tb': e.traceback or sys.exc_info()[2],
'expected': e.expected,
'cause': e.cause
}
'vcodec': {'type': 'ordered', 'regex': True,
'order': ['av0?1', 'vp0?9.2', 'vp0?9', '[hx]265|he?vc?', '[hx]264|avc', 'vp0?8', 'mp4v|h263', 'theora', '', None, 'none']},
'acodec': {'type': 'ordered', 'regex': True,
- 'order': ['opus', 'vorbis', 'aac', 'mp?4a?', 'mp3', 'e-?a?c-?3', 'ac-?3', 'dts', '', None, 'none']},
+ 'order': ['[af]lac', 'wav|aiff', 'opus', 'vorbis', 'aac', 'mp?4a?', 'mp3', 'e-?a?c-?3', 'ac-?3', 'dts', '', None, 'none']},
'hdr': {'type': 'ordered', 'regex': True, 'field': 'dynamic_range',
'order': ['dv', '(hdr)?12', r'(hdr)?10\+', '(hdr)?10', 'hlg', '', 'sdr', None]},
'proto': {'type': 'ordered', 'regex': True, 'field': 'protocol',
_ACCESS_TOKEN = None
_NETRC_MACHINE = 'fancode'
- _LOGIN_HINT = 'Use "--user refresh --password <refresh_token>" to login using a refresh token'
+ _LOGIN_HINT = 'Use "--username refresh --password <refresh_token>" to login using a refresh token'
headers = {
'content-type': 'application/json',
webpage)
def _extract_count(self, pattern, webpage, name):
- return str_to_int(self._search_regex(
- pattern, webpage, '%s count' % name, fatal=False))
+ return str_to_int(self._search_regex(pattern, webpage, '%s count' % name, default=None))
def _real_extract(self, url):
mobj = self._match_valid_url(url)
'series': 'Million Dollars, But...',
'episode': 'Million Dollars, But... The Game Announcement',
},
- 'skip_download': 'm3u8',
+ 'params': {'skip_download': True},
}, {
'url': 'https://roosterteeth.com/watch/rwby-bonus-25',
'info_dict': {
'thumbnail': r're:^https?://.*\.(png|jpe?g)$',
'ext': 'mp4',
},
- 'skip_download': 'm3u8',
+ 'params': {'skip_download': True},
}, {
'url': 'http://achievementhunter.roosterteeth.com/episode/off-topic-the-achievement-hunter-podcast-2016-i-didn-t-think-it-would-pass-31',
'only_matching': True,
elif username is not None:
self.report_warning(
'Login using username and password is not currently supported. '
- 'Use "--user oauth --password <oauth_token>" to login using an oauth token')
+ 'Use "--username oauth --password <oauth_token>" to login using an oauth token')
r'''
def genDevId():
from ..utils import (
ExtractorError,
smuggle_url,
+ str_or_none,
traverse_obj,
- unsmuggle_url,
unified_strdate,
+ unsmuggle_url,
)
import itertools
'id': voice_id,
'title': compat_str(value.get('PlaylistName')),
'uploader': value.get('SpeakerName'),
- 'uploader_id': compat_str(value.get('SpeakerId')),
+ 'uploader_id': str_or_none(value.get('SpeakerId')),
'channel': value.get('ChannelName'),
- 'channel_id': compat_str(value.get('ChannelId')),
+ 'channel_id': str_or_none(value.get('ChannelId')),
'upload_date': upload_date,
}
downloader.add_option(
'-N', '--concurrent-fragments',
dest='concurrent_fragment_downloads', metavar='N', default=1, type=int,
- help='Number of fragments of a dash/hlsnative video that should be download concurrently (default is %default)')
+ help='Number of fragments of a dash/hlsnative video that should be downloaded concurrently (default is %default)')
downloader.add_option(
'-r', '--limit-rate', '--rate-limit',
dest='ratelimit', metavar='RATE',
class MetadataFromFieldPP(MetadataParserPP):
@classmethod
def to_action(cls, f):
- match = re.match(r'(?P<in>.*?)(?<!\\):(?P<out>.+)$', f)
+ match = re.match(r'(?s)(?P<in>.*?)(?<!\\):(?P<out>.+)$', f)
if match is None:
raise ValueError(f'it should be FROM:TO, not {f!r}')
return (
write_string(
'DeprecationWarning: "yt_dlp.update.update_self" is deprecated and may be removed in a future version. '
- 'Use "yt_dlp.update.run_update(ydl)" instead')
+ 'Use "yt_dlp.update.run_update(ydl)" instead\n')
class FakeYDL():
_opener = opener
False if it has yet to be written out."""
# Adapted from http://stackoverflow.com/a/3259271/35070
- import ctypes
import ctypes.wintypes
WIN_OUTPUT_IDS = {
if codec in ('avc1', 'avc2', 'avc3', 'avc4', 'vp9', 'vp8', 'hev1', 'hev2',
'h263', 'h264', 'mp4v', 'hvc1', 'av1', 'theora', 'dvh1', 'dvhe'):
if not vcodec:
- vcodec = '.'.join(parts[:4]) if codec in ('vp9', 'av1') else full_codec
+ vcodec = '.'.join(parts[:4]) if codec in ('vp9', 'av1', 'hvc1') else full_codec
if codec in ('dvh1', 'dvhe'):
hdr = 'DV'
elif codec == 'av1' and len(parts) > 3 and parts[3] == '10':
hdr = 'HDR10'
elif full_codec.replace('0', '').startswith('vp9.2'):
hdr = 'HDR10'
- elif codec in ('mp4a', 'opus', 'vorbis', 'mp3', 'aac', 'ac-3', 'ec-3', 'eac3', 'dtsc', 'dtse', 'dtsh', 'dtsl'):
+ elif codec in ('flac', 'mp4a', 'opus', 'vorbis', 'mp3', 'aac', 'ac-3', 'ec-3', 'eac3', 'dtsc', 'dtse', 'dtsh', 'dtsl'):
if not acodec:
acodec = full_codec
else:
write_string('WARNING: Unknown codec %s\n' % full_codec, sys.stderr)
- if not vcodec and not acodec:
- if len(split_codecs) == 2:
- return {
- 'vcodec': split_codecs[0],
- 'acodec': split_codecs[1],
- }
- else:
+ if vcodec or acodec:
return {
'vcodec': vcodec or 'none',
'acodec': acodec or 'none',
'dynamic_range': hdr,
}
+ elif len(split_codecs) == 2:
+ return {
+ 'vcodec': split_codecs[0],
+ 'acodec': split_codecs[1],
+ }
return {}
projects / yt-dlp.git / commitdiff
