CyberD.org
C:\ Home » Code » My 'Custom Upload Dir' Story

My 'Custom Upload Dir' Story

Here's a little causerie in code about a little problem that's been hanging around in the back of my mind for a few years, that I finally did something about. If you're not working with or interested in WP, this might be a blog to skip reading. Short story, long blogpost, and it all starts...

A few years ago, I installed Custom Upload Dir. It's a one-of-a-kind plugin that lets you specify a custom upload folder within the base upload folder you've specified in WP, using a series of placeholders to modify directory structure based on things like file type, date, user, etc. I didn't need any of these extra modifiers, I just wanted to add an additional upload directory with a name of my choice for new files, but couldn't do this via WP since changing your upload directory directly by default changes the URL for all already uploaded images as well. So, I used this nifty plugin to start building a sequence of numerical sub-folders within the base directory that I'd switch between as they filled up over the years. On that note, it would be nice with a simpler plugin that automatically creates and shifts to a new directory when the current upload folder reaches a specified amount, creating this new directory based upon a naming pattern you specify. If any reader just so happens to catch onto the idea, let me know! I'd love to use it!

Anyway, I had previously used img/ as my base directory for all uploads, and was now planning to change this to work with the new directory pattern above. But since WP stores all meta-data links in relative form, I couldn't search and replace these links in my database with a new URL. There was just no URL to search for, and all images had unique filenames so there didn't seem to be a pattern I could use to search and find either. I thus changed my base directory to img/1/ and initiated my new practice of sub-folders within this sub-folder, leading to my images getting stored in img/1/, img/1/2/ and img/1/3/ and so on. I planned to change this later on, easily moving everything to the root folder, but I never figured out how.

Eventually, I changed the base directory back to img/, and moved all sub-folders into this directory, keeping the 1/ as just a sub-folder in the series. I searched and replaced all full URLs to these images in my post content, where the links were absolute, but couldn't do anything with them in post-meta, the table in which media library entries were stored. So since then they've worked in my posts, but not in the back-end where I browse these images.

A few days ago I decided on a whim to try this again. I opened up PhpMyAdmin not sure how I'd be able to accomplish this, but in exploring the database structure I quickly realized that all attachment images were stored in wp_attached_file entries within the wp_postmeta table. Wouldn't it be easy if I could just automatically search through the entries and add the folder number between table name and entry title? Aha. I exported wp_postmeta as an SQL file, opened it up in my favorite search and replace program (in this case TextRep 2.0, NoName Software), and searched for all wp_attached_file fields there were. I started the search and replace by searching for the first words in large series of images, like this:

wp_attached_file', 'One-Piece
wp_attached_file', '1/One-Piece

...but soon realized that using single letters would be much quicker! I searched by both upper and lowercase letters, like so:

wp_attached_file', 'O
wp_attached_file', '1/O
wp_attached_file', 'o
wp_attached_file', '1/o

...and 50 searches later, it was done!

You could probably do this via phpMyAdmin too, if you know how, but personally I found it much faster and easier to do it this way. Were my upload folder troubles entirely over? I looked through the postmeta file I'd downloaded and realized to my agonizing aggrevation that the thumbnail URLs remained unchanged. I had only changed the URLS to the images. Thumbs are stored in larger chunks of annoying text that look something like this:

a:6:{s:5:"width";s:3:"580";s:6:"height";s:3:"500";s:14:"hwstring_small";s:23:"height=''96'' width=''111''";s:4:"file";s:16:"One-Piece-40.jpg";s:5:"sizes";a:2:{s:9:"thumbnail";a:3:{s:4:"file";s:24:"One-Piece-40-200x200.jpg";s:5:"width";s:3:"200";s:6:"height";s:3:"200";}s:6:"medium";a:3:{s:4:"file";s:24:"One-Piece-40-280x241.jpg";s:5:"width";s:3:"280";s:6:"height";s:3:"241";}}s:10:"image_meta";a:10:{s:8:"aperture";s:1:"0";s:6:"credit";s:0:"";s:6:"camera";s:0:"";s:7:"caption";s:0:"";s:17:"created_timestamp";s:1:"0";s:9:"copyright";s:0:"";s:12:"focal_length";s:1:"0";s:3:"iso";s:1:"0";s:13:"shutter_speed";s:1:"0";s:5:"title";s:0:"";),

The really annoying thing with these chunks of text is that the numbers before the filenames for the thumbnails don't necessarily match. In one place it could be s:16, in another chunk s:18. It's a serialized array, where the numbers are apparently defined by the number of bits in each textual string. So, what did I do? What could I do? Nothing. I gave up. I changed the wp_attached_file URLs, but no wp_attachment_metadata data at all. I couldn't figure out how to do it, and neither could Google. I uploaded the postmeta SQL anyway to at least solve part of the issue aaand... it worked! The images worked. The thumbnails worked. Everything worked. As it turns out, I'd done all I needed to do!

To explain this totally unexpected surprise, I'm assuming either the wp_attached_file stores not only the filename (and relative URL if you are using a custom prefix as I am) but in fact the file info it's entirety, used as much for showing image info as for generating thumbnail info. That, or WP has changed how it handles this information and no longer generates thumbnail strings from uploaded images, relying on the image file to directly fetch such information when it's displayed. Of course I'm a total noob at this stuff and these are all very vague attempts at explaining this miracle. But the important thing is: that's all there is to it. :)

If I hadn't already started this blog ten years ago, I'd know till next time that how you structure the site from the ground-up is incredibly important. Choose a directory structure you like and stick to it, or make sure it's easy to change. Don't get lazy and stop naming, tagging and keeping track of files properly, and if you move the site: do an SQL export rather than a XML file export. Not that it's relevant to anything, but XML files don't store Media Library entries, so if you want to have a browsable gallery of all included images still available be weary of having them all wiped out! There are plugins for this, but none of them currently work with large directories and selections of files. But that's something for another post.

A couple of other notes I for some reason didn't include above:

If you're interested to know, I make a new subdirectory when it starts nearing 10k images. The Unix file system ext3 (which is supposedly the default nowadays) should be able to handle more files, but the web host or FTP client you use will still place further limits on how many files you can access or store in a single directory. Even ext2 supported 65,ooo files per directory, but unless you clearly know your limits on both hardware, software and host - and know that you won't change hosts later on - it can save some future aggregation dividing files into more manageable chunks.

I've previously tried storing more files in a single directory, but realized that suddenly they weren't showing up. 9998 files was the limit my host had imposed, and they weren't willing to change it.

If you're connecting via FTP and all files aren't displayed, try SFTP, that usually allows for more. On the other hand SFTP is a secure(r) protocol and thus takes significantly longer to load a directory.

Also, why doesn't WP allow you to store uploads with a structure of your own choosing anyway? Just one large folder or folders sorted by upload year and month surely isn't the most ideal. I've never liked sorting anything other than my photo folder by date, since it's just difficult to keep track of all the different folders, and with WP you get a new folder for each month even if you only upload a single image that month. Overkill.

That's all.

Comments

Keep track of the discussion via rss? Read about comment etiquette? Or type in something below!
This was pretty damn interesting. And yet, nobody's spoken! Be the first!


The Comment Form

Your email address will not be published. Required fields are marked *

Your email is saved only to approve your future comments automatically (assuming you really are a human). ;) It's not visible or shared with anyone. You can read about how we handle your info here.

Question   Smile  Sad   Redface  Biggrin  Surprised   Eek  Confused  Beardguy  Baka  Cool  Mad   Twisted  Rolleyes   Wink  Coin

Privacy   Copyright   Sitemap   Statistics   RSS Feed   Valid XHTML   Valid CSS   Standards

© CyberD.org 2025
Keeping the world since 2004.