Migrating from 32 to 64 bits (going from Debian i386 to amd64)

The new version of Debian, due in some days (Debian 7.0, Wheezy) comes with multiarch support, which doesn't officially support migrating from 32 to 64 bits (at least I think so), but I have managed to migrate from i386 to amd64 with just one reboot to change kernels.

This is something that is not officially supported and that shouldn't be done if you don't have enough knowledge of the Debian system, as you can have some problems that may need some manual things done. I'll try to explain what I did but this is not a step by step guideline and is prone to fail in most cases needing you to install things by hand or similar things.

One more warning... you should be on Wheezy before trying to do any of this, trying to go from Squeeze i386 to Wheezy amd64 has given me some heavy problems, I also managed to do it but with too much manual intervention.

As always, I'm not liable of any problem you may have trying to do any of this.

The first thing I did as the machine was a pre i386 machine with an i386 kernel, was to install linux-image-amd64 in order to have a 64 bits kernel, this kernel will allow us to run both i386 and amd64 apps. Then I rebooted in order to run this kernel. It may be a good idea to install some extra packages like deborphan and binutils just in case we may need them later. To allow us to install the amd64 packages we must enable the arch by using:
dpkg --add-architecture amd64 apt-get update After that we can start installing the amd64 packages, what I did was: apt-get install libc6:amd64 And then apt-get install apt:amd64 There I needed to answer (Yes, do as I say!) as we are installing apt and this will uninstall apt which is something you wouldn't usually want, but we want it here as we are installing apt :-)

Then I went for a
apt-get install dpkg:amd64 even though on some other tests dpkg seemed to be amd64 already after the apt installation, so maybe this is not needed. Anyway, after that it looks like doing a apt-get -f install installs all the amd64 basic dependencies. But it seems that this command can break sometimes, or remove some packages on other times, what I did was note down the removed stuff in order to install them afterwards (remember that this will only remove the binaries, the config files will stay there so if you install them later they'll inherit that config, which is what we want). I had to run this command several times until I got it to end.

Getting this ended doesn't mean that we are done. Doing a dpkg -l|grep :i386 will let us know what packages still remain on i386 arch. You can install them again with apt and they should be reinstalled on amd64 arch. After that you can install all the stuff that had previously been removed and afterwards do some cleaning with apt-get autoremove --purge If you have finished cleaning up using deborphan or whatever and you do not see any i386 files on dpkg... you are done :-)



TROUBLESHOOTING:

I had a problem with findutils as it seems that Debian dpkg system depends on find a lot when it tries to install findutils. To solve this I went to /var/cache/apt/archives/ and did dpkg -x findutils... /tmp/findutils and then had to cp the find binary to its place and then dpkg -i findutils... I had similar problem with bash and dash which go together, again I had to apply the same solution as on the findutils case.

On some heavy problems you may end up with dpkg not working at all, you can then use ar x package.deb to extract that data.tar.?z and then tar to extract the binaries of the package out if there.

A useful command is dpkg -l|grep ^rc to see the packages that have been removed but still have configs around.

This is something I used to try to identify targets to install: deborphan -a -n|sed -n "s/[^ ]* *\(.*\):i386/\1:amd64/p"

Getting a file out of your squid cache

I suppose there may be some tools out there to do this, maybe even the squidclient can do this, but hey... doing it with a script is always funnier, and learning how squid does things is great, so... The first thing we must do is to identify our target, if I was to try to get the Android Quadrant Standard apk file out of my squid cache I should first look at my (I'm assuming Debian paths, as always) /var/log/squid/store.log* and look for it. As a hint, if you look for android apks you sould be looking for application/vnd.android.package-archive (which is the mime type they use). You should find something like this: 1366393264.443 SWAPOUT 00 00009545 29D0FF2BA5A2F31424F3C49102C91657 200 1366364657 1339281848 2147368447 application/vnd.android.package-archive 1452291/1452291 GET http://r14---sn-4g57lne7.c.android.clients.google.com/market/GetBinary/com.aurorasoftworks.quadrant.ui.standard/2010100?ms=au&... And in here the fourth entry is the file name under which squid stored the content it has cached, so the path for this file would be /var/spool/squid/sedond_byte_of_name/third_byte_of_name, in this case... /var/spool/squid/00/95/00009545 But you won't even need to compute that, just get that fourth entry (yes, that weird number) and type it on this little script... read filename; file="/var/spool/squid/${filename:2:2}/${filename:4:2}/$filename"; file_length=$(ls -l $file|cut -d " " -f5); content_length=$(head $file|sed -n "s/Content-Length: \(.*\)\r$/\1/p"); dd if=$file bs=$(($file_length-$content_length)) skip=1 of=/tmp/wanted.apk and you'll get at /tmp/wanted.apk the file you wanted. The script can be written on one line without spaces if you want to have it as an alias or whatever, I just inserted the lines to make it more readable on the web. The logic on the script makes the full pathname of the file from the filename you type (followed by return) on its input, then gets the length of that squid file and the content length (which is the length of the apk) from the header of the squid file, then uses these two values to compute the length of the squid header and skips it on dd copying the rest of it to /tmp/wanted.apk. BTW... you may be wondering why the \r$ on the sed expresion... yes, there seems to be a \r at the end of the Content-Length line, don't know why, but it is there, at least on my system That's it!

Making squid cache your downloads (android apk or anything you want)

Squid has some patterns that avoid caching, like when something is a query to a search engine or you are calling a cgi, ... this can avoid caching certain things that you may like to have cached, like for example the apks you download from google play, but of course you can make it cache them if you want. First just a note for people wanting to add transparent proxy to squid, at squid you should put the listening http port on transparent mode (or intercept on recent squids), something like: http_port 3128 transparent And then you must take to squid the traffic you want it to process, this is typically done using iptables with something like this: iptables -I PREROUTING -i wifi0 -s my_android_ip -p tcp --dport 80 -m addrtype ! --dst-type LOCAL -j REDIRECT --to-port 3128 With this two things done you should have the traffic you want being processed by your squid and you should be able to see it at /var/log/squid/access.log, but this doesn't mean it is caching what you want it to cache, for that we should find something that identifies the things we want to cache, like the url, for example, I chose this for google play apk downloads: acl market url_regex http://[^/]*\.android\.clients\.google\.com And then we tell squid that we want that cached: cache allow market That should make it, but I recommend you add also at least this couple of lines if you don't have them already, these are the lines to blame for not caching everything, but should be on your config anyway: acl QUERY urlpath_regex cgi-bin \? cache deny QUERY And that's it. I'll try to write how to extract that apk from the cache in a while, see you! ;-)