WordPress import/export design annoyance

WordPress doesn’t embed media in BASE64 format in it’s XML like it should and instead embeds the absolute path and prompts the user for the files upon import. What’s more is they make actual XML nodes for everything in your media library as attachment post types, but it’s only meta data.. God forbid there be 50KB-11MB blobs in the XML..

I wrote a python script that basically does was WP export does with all-content and is compatible with the WP import system. It works on offline SQL dumps and media directories though(as is my case).. I couldn’t automate media import because of this, though..

 

Advertisements

Update and FFMPEG HLS ripping

I made a post about this on my old blog, but I’ve used it a while since, and I found out I have to recover data from the TAR GoDaddy created unless I can get another unpacker to work, and since I’m slightly lazy and slightly busy it hasn’t been done yet(even though the tools are ready).

I’ll merge below new posts what I manage to recover from the SQL table in that TAR, later..

Basically, most big streaming vendors out there in 2018 are using HLS(HTTP Live Streaming) with or without AES 128bit CBC segment encryption, and with or without session timeouts on embedded segment key-URI. By “big” I mean like Netflix, Discovery Network(science channel etc..), most podcast networks with video etc..

I suggest the 64bit static nightly portable builds from ffmpeg.org for this work.

You typically need a packet sniffer that can watch TLS HTTP streams(sadly I don’t think any use local root store and hooks and just handle it transparently locally; you’d need sslsplit which can do HSTS MITM(but not pinning) etc..) or a Dev Tools like option in your browser that monitors network resources like Chrome has under F12.

You need to watch for the very first m3u8 in your HTTP stream using whichever method, and once you have that depending on if the host uses session timeouts on root(AES key in root m3u8 that decrypts segment keys; only used some places) and segment keys, then you may need to act hastily.

The command:

ffmpeg -protocol_whitelist https,tls,tcp,file -i <full path to the root m3u8> -c copy -map 0:p:2 <output file name>.mp4

You’ll notice the -map 0:p:2. Most root m3u8 have multiple videos and a audio option like the following(minus the audio option but it’s basically the same as video, and minus a a root AES key used to decrypt segment keys) :

#EXTM3U
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1927168,RESOLUTION=1280×720
<child m3u8 URL HERE>
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=514048,RESOLUTION=640×360
<child m3u8 URL HERE>
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=2277376,RESOLUTION=1280×720
<child m3u8 URL HERE>

That looks like this to FFMPEG:

Untitled.png

So.. -map 0:p:2 tells FFMPEG we want input 0, “Program 2”

To finish this up. Here is how to have FFMPEG convert to another video codec in the same command with full hardware acceleration:

ffmpeg -hwaccel auto -protocol_whitelist https,tls,tcp,file -i <full path to the root m3u8> -c:a copy -c:v libx265 -map 0:p:2 <output file name>.mp4

Replace “libx265” with anything you see in “-codecs”. To me 265/HEVC is the best thing out there till AOM AV1 is finished in terms of size:quality efficiency.

Use this to see a list of hardware accelerations on your system

ffmpeg -hwaccels