Caching FreeBSD packages with a proxy server
2015 February 21
By setting up apt-cacher-ng 1 in you local development environment, you can save on precious bandwidth when building Chef recipes (or any server side stuff) on your local machine. Below are some of the steps to get this up and running.
Install and configure apt-cacher-ng
In a Mac you can get it via homebrew
brew install apt-cacher-ng
When configuring apt-cacher-ng you need to set the following
- Setup the freebsd repository
- Set Pigeonhole files for *.txz files for packages and meta.txz, packagesite.txz, digests.txz for metadata files.
TLDR: Place these changes to your vanilla apt-cacher-ng configuration.
--- acng.conf.vanilla 2015-02-19 23:50:34.000000000 +0900
+++ acng.conf 2015-02-21 01:40:51.000000000 +0900
@@ -62,6 +62,7 @@
Remap-epel: file:epel_mirrors # Fedora EPEL
Remap-slrep: file:sl_mirrors # Scientific Linux
Remap-gentoo: file:gentoo_mirrors.gz /gentoo ; file:backends_gentoo # Gentoo Archives
+Remap-freebsd: pkg.freebsd.org pkgmir.pkg.freebsd.org
# This is usually not needed for security.debian.org because it's always the
# same DNS hostname. However, it might be enabled in order to use hooks
@@ -170,8 +171,8 @@
# Pigeonholing files with regular expressions (static/volatile). Can be
# overriden here but not should not be done permanently because future update
# of default settings would not be applied later.
-# VfilePattern = (^|.*/)(Index|Packages(\.gz|\.bz2|\.lzma|\.xz)?|InRelease|Release|Release\.gpg|custom\.gpg|mirrors.txt|Sources(\.gz|\.bz2|\.lzma|\.xz)?|release|index\.db-.*\.gz|Contents-[^/]*(\.gz|\.bz2|\.lzma|\.xz)?|pkglist[^/]*\.bz2|rclist[^/]*\.bz2|meta-release[^/]*|Translation[^/]*(\.gz|\.bz2|\.lzma|\.xz)?|MD5SUMS|SHA1SUMS|((setup|setup-legacy)(\.ini|\.bz2|\.hint)(\.sig)?)|mirrors\.lst|repo(index|md)\.xml(\.asc|\.key)?|directory\.yast|products|content(\.asc|\.key)?|media|filelists\.xml\.gz|filelists\.sqlite\.bz2|repomd\.xml|packages\.[a-zA-Z][a-zA-Z]\.gz|info\.txt|license\.tar\.gz|license\.zip|.*\.(db|files|abs)(\.tar(\.gz|\.bz2|\.lzma|\.xz))?|metalink\?repo|.*prestodelta\.xml\.gz|repodata/.*\.(xml|sqlite)(\.gz|\.bz2|\.lzma|\.xz))$|/dists/.*/installer-[^/]+/[^0-9][^/]+/images/.*
-# PfilePattern = .*(\.d?deb|\.rpm|\.drpm|\.dsc|\.tar(\.gz|\.bz2|\.lzma|\.xz)(\.gpg)?|\.diff(\.gz|\.bz2|\.lzma|\.xz)|\.jigdo|\.template|changelog|copyright|\.udeb|\.debdelta|\.diff/.*\.gz|(Devel)?ReleaseAnnouncement(\?.*)?|[a-f0-9]+-(susedata|updateinfo|primary|deltainfo).xml.gz|fonts/(final/)?[a-z]+32.exe(\?download.*)?|/dists/.*/installer-[^/]+/[0-9][^/]+/images/.*)$
+VfilePattern = (^|.*/)((meta|packagesite|digests)\.txz|Index|Packages(\.gz|\.bz2|\.lzma|\.xz)?|InRelease|Release|Release\.gpg|custom\.gpg|mirrors.txt|Sources(\.gz|\.bz2|\.lzma|\.xz)?|release|index\.db-.*\.gz|Contents-[^/]*(\.gz|\.bz2|\.lzma|\.xz)?|pkglist[^/]*\.bz2|rclist[^/]*\.bz2|meta-release[^/]*|Translation[^/]*(\.gz|\.bz2|\.lzma|\.xz)?|MD5SUMS|SHA1SUMS|((setup|setup-legacy)(\.ini|\.bz2|\.hint)(\.sig)?)|mirrors\.lst|repo(index|md)\.xml(\.asc|\.key)?|directory\.yast|products|content(\.asc|\.key)?|media|filelists\.xml\.gz|filelists\.sqlite\.bz2|repomd\.xml|packages\.[a-zA-Z][a-zA-Z]\.gz|info\.txt|license\.tar\.gz|license\.zip|.*\.(db|files|abs)(\.tar(\.gz|\.bz2|\.lzma|\.xz))?|metalink\?repo|.*prestodelta\.xml\.gz|repodata/.*\.(xml|sqlite)(\.gz|\.bz2|\.lzma|\.xz))$|/dists/.*/installer-[^/]+/[^0-9][^/]+/images/.*
+PfilePattern = .*(\.d?deb|\.rpm|\.drpm|\.txz|\.dsc|\.tar(\.gz|\.bz2|\.lzma|\.xz)(\.gpg)?|\.diff(\.gz|\.bz2|\.lzma|\.xz)|\.jigdo|\.template|changelog|copyright|\.udeb|\.debdelta|\.diff/.*\.gz|(Devel)?ReleaseAnnouncement(\?.*)?|[a-f0-9]+-(susedata|updateinfo|primary|deltainfo).xml.gz|fonts/(final/)?[a-z]+32.exe(\?download.*)?|/dists/.*/installer-[^/]+/[0-9][^/]+/images/.*)$
#
# Whitelist for expiration, file types not to be removed even when being
# unreferenced. Default: many parts from VfilePattern where no parent index
/usr/local/etc/apt-cacher-ng$ vagrant plugin list
Finally run apt-cacher-ng like as follows on your mac
launchctl load /usr/local/opt/apt-cacher-ng/homebrew.mxcl.apt-cacher-ng.plist
Update your base vagrant box
I am using Chef’s FreeBSD 10.1 basebox 2 from its bento repository 3 here. I modify the Vagrantfile in the basebox as follows.
Vagrant.configure '2' do |config|
config.vm.provision 'shell', inline: <<-eos
#!/bin/sh
mkdir -p /usr/local/etc/pkg/repos
cat > /usr/local/etc/pkg/repos/freebsd.conf <<EOF
FreeBSD { url: "pkg+http://<ip-of-mac-vagrant-host>:3142/pkg.FreeBSD.org/\\${ABI}/latest" }
EOF
eos
end
The basebox’ Vagrantfile is found in ~/.vagrant.d/boxes/chef-VAGRANTSLASH-freebsd10.1/0/vmware_desktop/Vagrantfile
in my case.