With the current setup, if no maintenance is done, the log files from lighhtpd will just keep on growing. OpenWRT itself is configured to rotate its log files once they grow beyond 1MB. This is pretty nice, and since webalizer is configured to cope with disappearing log entries, it would be nice to rotate the web server log as well. To do this, install logrotate.

opkg install logrotate

logrotate has a global configuration file in /etc/logrotate.conf, and for individual programs in the directory /etc/logrotate.d/. Read the man page at Ubuntu Manpage: logrotate on the use of logrotate and the syntax of the configuration files. To rotate the log files of lighttpd, add the file /etc/logrotate.d/lighttpd, with the following contents:

/mnt/data/log/lighttpd/access.log {
    weekly
    rotate = 53
    su http www-data
    postrotate
        /usr/bin/killall -HUP  lighttpd        
    endscript
}

This tells logrotate to rotate lighttpd logs, every week, and keep about a years worth of log files. The log files needs to be accessed as user http, and group www-data, and lighttpd is told to create a new log file by sending it the SIGHUP signal.

logrotate needs to be run daily, so set up a cron job for it. Add the following to run logrotate once a day at 8:00.

0 8 * * * *     /usr/sbin/logrotate -s /mnt/data/tmp/logrotate.state /etc/logrotate.conf

Installing an updated OpenWRT image.

OpenWrt Sysupgrade

OpenWRT has a command, sysupgrade, that is used to upgrade the flash image from an update file. I have not been able to get this to work, and have ended up installing LUCI, the web configuration interface, every time I need to update the root file system.

opkg install luci

This opens another can of worms, since lighttpd is happily serving pages on port 80, where LUCIs web server, uHTTPd, wants to be. To get around this, I told uHTTPd to use some other ports. To do this, change the ports in the lines containing listen in /etc/config/uhttpd, like so:

config uhttpd 'main'
    list listen_http '0.0.0.0:8080'
    list listen_http '[::]:8080'
    list listen_https '0.0.0.0:4430'
    list listen_https '[::]:4430'

Then restart uHTTPd:

/etc/init.d/uhttpd restart

LUCI will now be available on the current IP address, on port 8080 and encrypted on 4430. Use the root user/password to login in, and use openwrt-oxnas-stg212-ubifs-sysupgrade.tar to update the device.

After flashing the firmware, all packages need to be reinstalled. Opkg will probably complain about changed config files, but this just means our configuration changes have been kept.

The webalizer is a program, that scans web server log files, and creates statistics from the entries. Since webalizer uses log files, there are no cookies involved, and webalizer uses flat files for storage, no database to install. Webalizer is designed to be run at some interval to update the statistics.

Installing and configuring.

opkg install webalizer

To run webalizer, create a cron job for it in /etc/crontab/http, by putting the job in the http file, cron is told to run it as the http user.

0 */12 * * * /usr/bin/webalizer -q

There is a thoroughly commented config file in /etc/webalizer.conf.sample, to use this as a the base for your changes, copy the file to /etc/webalizer.conf.

cp /etc/webalizer.conf.sample /etc/webalizer.conf

Here is my configuration:

#
# Webalizer configuration file
#

# LogFile defines the web server log file to use.  If not specified
# here or on on the command line, input will default to STDIN.  If
# the log filename ends in '.gz' (a gzip compressed file), or '.bz2'
# (bzip2 compressed file), it will be decompressed on the fly as it
# is being read.

LogFile        /mnt/data/log/lighttpd/access.log

# OutputDir is where you want to put the output files.  This should
# should be a full path name, however relative ones might work as well.
# If no output directory is specified, the current directory will be used.

OutputDir      /mnt/data/www/stats

# HistoryName allows you to specify the name of the history file produced
# by the Webalizer.  The history file keeps the data for previous months,
# and is used for generating the main HTML page (index.html). The default
# is a file named "webalizer.hist", stored in the output directory being
# used.  The name can include a path, which will be relative to the output
# directory unless absolute (starts with a leading '/').

HistoryName webalizer.hist

# Incremental processing allows multiple partial log files to be used
# instead of one huge one.  Useful for large sites that have to rotate
# their log files more than once a month.  The Webalizer will save its
# internal state before exiting, and restore it the next time run, in
# order to continue processing where it left off.  This mode also causes
# The Webalizer to scan for and ignore duplicate records (records already
# processed by a previous run).  See the README file for additional
# information.  The value may be 'yes' or 'no', with a default of 'no'.
# The file 'webalizer.current' is used to store the current state data,
# and is located in the output directory of the program (unless changed
# with the IncrementalName option below).  Please read at least the section
# on Incremental processing in the README file before you enable this option.

Incremental yes

# IncrementalName allows you to specify the filename for saving the
# incremental data in.  It is similar to the HistoryName option where the
# name is relative to the specified output directory, unless an absolute
# filename is specified.  The default is a file named "webalizer.current"
# kept in the normal output directory.  If you don't specify "Incremental"
# as 'yes' then this option has no meaning.

IncrementalName webalizer.current

# HostName defines the hostname for the report.  This is used in
# the title, and is prepended to the URL table items.  This allows
# clicking on URLs in the report to go to the proper location in
# the event you are running the report on a 'virtual' web server,
# or for a server different than the one the report resides on.
# If not specified here, or on the command line, webalizer will
# try to get the hostname via a uname system call.  If that fails,
# it will default to "localhost".

HostName    groenholdt.net

# PageType lets you tell the Webalizer what types of URLs you
# consider a 'page'.  Most people consider html and cgi documents
# as pages, while not images and audio files.  If no types are
# specified, defaults will be used ('htm*', 'cgi' and HTMLExtension
# if different for web logs, 'txt' for ftp logs).

PageType    htm*
PageType    cgi
#PageType   phtml
#PageType   php3
#PageType   pl

# OmitPage lets you tell the Webalizer that certain URLs do not
# contain any pages.  No URL matching an OmitPage value will be
# counted as a page, even if it matches a PageType above or has
# no extension (e.g., a directory).  They will still be counted
# as a hit.

OmitPage    /stats/

# UseHTTPS should be used if the analysis is being run on a
# secure server, and links to urls should use 'https://' instead
# of the default 'http://'.  If you need this, set it to 'yes'.
# Default is 'no'.  This only changes the behaviour of the 'Top
# URLs' table.

#UseHTTPS       no

# DNSCache specifies the DNS cache filename to use for reverse DNS lookups.
# This file must be specified if you wish to perform name lookups on any IP
# addresses found in the log file.  If an absolute path is not given as
# part of the filename (ie: starts with a leading '/'), then the name is
# relative to the default output directory.  See the DNS.README file for
# additional information.

DNSCache    dns_cache.db

# DNSChildren allows you to specify how many "children" processes are
# run to perform DNS lookups to create or update the DNS cache file.
# If a number is specified, the DNS cache file will be created/updated
# each time the Webalizer is run, immediately prior to normal processing,
# by running the specified number of "children" processes to perform
# DNS lookups.  If used, the DNS cache filename MUST be specified as
# well.  The default value is zero (0), which disables DNS cache file
# creation/updates at run time.  The number of children processes to
# run may be anywhere from 1 to 100, however a large number may affect
# normal system operations.  Reasonable values should be between 5 and
# 20.  See the DNS.README file for additional information.

DNSChildren 8

# CacheIPs allows unresolved IP addresses to be cached in the DNS
# database.  Normally, only resolved addresses are saved.  At some
# sites, particularly those with a large number of unresolvable IP
# addresses visiting, it may be useful to enable this feature so
# those addresses are not constantly looked up each time the program
# is run.  Values can be 'yes' or 'no', with 'no' being the default.

CacheIPs    yes

# CacheTTL specifies the time to live (TTL) value for cached DNS
# entries, in days.  This value may be anywhere between 1 and 100
# with the default being 7 days (1 week).

#CacheTTL   7

# The GeoDB option enables or disabled the use of the native
# Webalizer GeoDB geolocation services.  This is the preferred
# geolocation option.  Values may be 'yes' or 'no', with 'no'
# being the default.

#GeoDB      no

# GeoDBDatabase specifies an alternate database to use.  The
# default database is /usr/share/GeoDB/GeoDB.dat (however the
# path may be changed at compile time; use the -vV command
# line option to determine where).  If a different database is
# to be used, it may be specified here.  The name is relative
# to the output directory being used unless an absolute name
# (ie: starts with a leading '/') is specified.

#GeoDBDatabase  /usr/share/GeoDB/GeoDB.dat

# The GeoIP option enables or disables the use of geolocation
# services provided by the GeoIP library (http://www.maxmind.com),
# if available.  Values may be 'yes' or 'no, with 'no' being the
# default.  Note: if GeoDB is enabled, then this option will have
# no effect (GeoDB will be used regardless of this setting).

#GeoIP      no

# GeoIPDatabase specifies an alternate database filename to use by the
# GeoIP library.  If an absolute path is not given as part of the name
# (ie: starts with a leading '/'), then the name is relative to the
# default output directory. This option should not normally be needed.

#GeoIPDatabase  /usr/share/GeoIP/GeoIP.dat

# VisitTimeout allows you to set the default timeout for a visit
# (sometimes called a 'session').  The default is 30 minutes,
# which should be fine for most sites.
# Visits are determined by looking at the time of the current
# request, and the time of the last request from the site.  If
# the time difference is greater than the VisitTimeout value, it
# is considered a new visit, and visit totals are incremented.
# Value is the number of seconds to timeout (default=1800=30min)

#VisitTimeout   1800

# The All* keywords allow the display of all URLs, Sites, Referrers
# User Agents, Search Strings and Usernames.  If enabled, a separate
# HTML page will be created, and a link will be added to the bottom
# of the appropriate "Top" table.  There are a couple of conditions
# for this to occur..  First, there must be more items than will fit
# in the "Top" table (otherwise it would just be duplicating what is
# already displayed).  Second, the listing will only show those items
# that are normally visable, which means it will not show any hidden
# items.  Grouped entries will be listed first, followed by individual
# items.  The value for these keywords can be either 'yes' or 'no',
# with the default being 'no'.  Please be aware that these pages can
# be quite large in size, particularly the sites page,  and separate
# pages are generated for each month, which can consume quite a lot
# of disk space depending on the traffic to your site.

#AllSites   no
#AllURLs    no
#AllReferrers   no
#AllAgents  no
AllSearchStr    yes
#AllUsers       no

# The Hide*, Group* and Ignore* and Include* keywords allow you to
# change the way Sites, URLs, Referrers, User Agents and Usernames
# are manipulated.  The Ignore* keywords will cause The Webalizer to
# completely ignore records as if they didn't exist (and thus not
# counted in the main site totals).  The Hide* keywords will prevent
# things from being displayed in the 'Top' tables, but will still be
# counted in the main totals.  The Group* keywords allow grouping
# similar objects as if they were one.  Grouped records are displayed
# in the 'Top' tables and can optionally be displayed in BOLD and/or
# shaded. Groups cannot be hidden, and are not counted in the main
# totals. The Group* options do not, by default, hide all the items
# that it matches.  If you want to hide the records that match (so just
# the grouping record is displayed), follow with an identical Hide*
# keyword with the same value.  (see example below)  In addition,
# Group* keywords may have an optional label which will be displayed
# instead of the keywords value.  The label should be separated from
# the value by at least one 'white-space' character, such as a space
# or tab.  If the match string contains whitespace (spaces or tabs),
# the string should be quoted with either single or double quotes.
#
# The value can have either a leading or trailing '*' wildcard
# character.  If no wildcard is found, a match can occur anywhere
# in the string. Given a string "www.yourmama.com", the values "your",
# "*mama.com" and "www.your*" will all match.

# Your own site should be hidden
HideSite    *groenholdt.net
HideSite    localhost

# Your own site gives most referrals
HideReferrer    groenholdt.net/

# Usually you want to hide these
HideURL     *.gif
HideURL     *.GIF
HideURL     *.jpg
HideURL     *.JPG
HideURL     *.png
HideURL     *.PNG
HideURL     *.ra

# The MangleAgents allows you to specify how much, if any, The Webalizer
# should mangle user agent names.  This allows several levels of detail
# to be produced when reporting user agent statistics.  There are six
# levels that can be specified, which define different levels of detail
# supression.  Level 5 shows only the browser name (MSIE or Mozilla)
# and the major version number.  Level 4 adds the minor version number
# (single decimal place).  Level 3 displays the minor version to two
# decimal places.  Level 2 will add any sub-level designation (such
# as Mozilla/3.01Gold or MSIE 3.0b).  Level 1 will attempt to also add
# the system type if it is specified.  The default Level 0 displays the
# full user agent field without modification and produces the greatest
# amount of detail.  User agent names that can't be mangled will be
# left unmodified.

#MangleAgents    0

# The SearchEngine keywords allow specification of search engines and
# their query strings on the URL.  These are used to locate and report
# what search strings are used to find your site.  The first word is
# a substring to match in the referrer field that identifies the search
# engine, and the second is the URL variable used by that search engine
# to define its search terms.

#SearchEngine   .google.    q=
#SearchEngine   yahoo.com   p=
#SearchEngine   altavista.com   q=
#SearchEngine   aolsearch.      query=
#SearchEngine   ask.co          q=
#SearchEngine   eureka.com  q=
#SearchEngine   lycos.com   query=
#SearchEngine   hotbot.com  MT=
#SearchEngine   msn.com     q=
#SearchEngine   infoseek.com    qt=
#SearchEngine   excite      search=
#SearchEngine   netscape.com    query=
#SearchEngine   mamma.com   query=
#SearchEngine   alltheweb.com   q=
#SearchEngine   northernlight.com  qr=

# End of configuration file...  Have a nice day!

These are important options:

  • OutputDir /mnt/data/www/stats
  • Incremental yes Save state information, so that webalizer will not forget, even if the logs are cleared.
  • HostName groenholdt.net The host name of your server.
  • OmitPage /stats/ Do not keep statistics of the statistics.
  • HideSite *groenholdt.net
  • HideSite localhost
  • HideReferrer groenholdt.net/ Hide access from oneself from the "Top" lists.

Make the configuration file readable to all users.

chmod a+r /etc/webalizer.conf

Create the directories, and set user http, group www-data as owner. Make the directory writable by all members of the group www-data. This is done so that both lighttpd and The Webalizer have permission to access the files.

mkdir -p /mnt/data/www/stats/
chown http:www-data /mnt/data/www/stats -R
chmod g+w /mnt/data/www/stats -R

To restrict access to webalizer to computers on the local network, add these lines to /etc/lighttpd/lighttpd.conf:

# Deny the access to statistics to all user which 
# are not in the 192.168.0.x network
$HTTP["remoteip"] !~ "^(192\.168\.0\.)" {
  $HTTP["url"] =~ "^/stats/" {
    url.access-deny = ( "" )
  }
}

Replace "^(192\.168\.0\.)" with an IP address matching your network.

After getting Gentoo to run on the Medion NAS in these posts,

I learned that OpenWRT had been ported to the oxnas platform. This appealed to me, as OpenWRT is installed on the internal flash. Honestly the Gentoo installation I had on the HDD, was better suited as a web server, but I just wanted to play with OpenWRT. The only real reason for using OpenWRT instead of Gentoo, is in the hopes that the OpenWRT folks will keep the kernel updated.

I still have my web page, and the files from which it is generated, on a hard disk drive, connected to the SATA port.

I recommend having a serial connection to the NAS running at all times.

Installing OpenWRT bootstrap.

This operation can only be executed once, and may brick your device, after the bootstrap there is no way to restore the original firmware. Because of this I can not actually check that these steps are exactly right, but they are what i recall.

To bootstrap the installation I used the binary image from Gitorious openwrt-oxnas.

Setup a HTTP server on a computer to serve openwrt-oxnas-stg212-u-boot-initramfs.itb. Telnet into the NAS using the backdoor described on mikrocontroller.net. Login to the web-interface on the NAS, then open http://(NAS IP)/r36807,/adv,/cgi-bin/remote_help-cgi?type=backdoor (you may have to replace the /rXXXXX,/ with the revision number shown in the URL after login). The browser will wait for the CGI script to (never) end, while it’s doing that telnet into the NAS. Login with user root and the password also used by the web interface (default is 1234).

telnet (NAS IP)

After logging in, download the OpenWRT image to /tmp/tmpfs. Look in /proc/mtd and make sure kernel is in /dev/mtd4. Write the image to /dev/mtd4, tell U-Boot to boot from it, and reboot.

cd /tmp/tmpfs wget http://(server IP)/openwrt-oxnas-stg212-u-boot-initramfs.itb
cat /proc/mtd 
nandwrite /dev/mtd4 openwrt-oxnas-stg212-u-boot-initramfs.itb 
fw_setenv boot_stage2 nand read 64000000 440000 90000\\; go 64000000
fw_setenv bootcmd run boot_stage2 reboot

This is where the serial connection comes in handy, for watching the boot process. If everything went well LUCI, OpenWRT's web interface should be available on the NAS on address 192.168.1.1. When you have compiled a new OpenWRT image you can flash it, by using LUCI.

Building OpenWrt.

OpenWrt Buildroot – About.

Since, for now, oxnas support is only in OpenWRT trunk, everything needs to be build.

I build this on a Gentoo system, which seems to need automake-1.14 installed for glib2 to build.

Getting the sources.

Change into the directory where you want the sources to reside and do:

git clone git://git.openwrt.org/openwrt.git 
cd openwrt

OpenWRT uses

To have the standard set of packages available for OpenWRT copy feeds.conf.default to feeds.conf.

cp feeds.conf.default feeds.conf

Custom feeds.

If you just want a web server, and do not need setuptools for Python 3, or my shiny site generator, you can skip this step.

I have made a couple of custom feeds, that addresses some specific Python 3 needs I have for my static site generator. To have these packages available add the following to feeds.conf:

src-git packages https://github.com/deadbok/packages.git
src-git deadbok https://github.com/deadbok/deadbok-openwrt.git

Comment out the original package line in the file.

Update and add the feeds.

Add the packages to the build system.

./scripts/feeds update -a
./scripts/feeds install -a

Configuring the sources.

I have configured a lot of stuff, that I am not using right now, as modules, so that I can later install them if I find a need. This increases the build time, so it is a trade off compared to building just the packages that you want right now. You can download my configuration file, and use it as a basis for your own configuration.

wget http://groenholdt.net/Computers/OpenWRT/openwrt-config
mv openwrt-config .config

To configure the OpenWRT build run make menuconfig in the source directory.

I can not describe every configuration option, but here are some important ones.

First to build OpenWRT for the NAS these tell the build system about the basic hardware:

Target System (PLXTECH/Oxford NAS782x/OX82x)
Target Profile (MitraStar STG-212)

Under Target Images select

  • ubifs is the file system of the images we will be building for the NAS.

  • ramdisk I always build a RAM disk as well, since it can be used to unbrick the device, if you can still access the boot loader through the serial connection. Under Target Imaqes -> ramdisk make sure xz compression is selected.

Under Global build settings I enable at least

  • Enable shadow password support to have encrypted passwords for users in /etc/shadow.

  • Support for paging of anonymous memory (swap) To enable swap functionality in the kernel.

  • I disable all kernel debugging features, as this is a production environment.

If you want to develop or debug the build process of packages in OpenWRT enable Advanced configuration options (for developers), some sub-options that I use are:

  • Automatic rebuild of packages rebuilds packages when their files changes.

  • Enable log files during build process log build output in files under log/.

Under Base system

  • ca-certificates build as a module for STFP, HTTPS etc.

  • Enable the firewall as you might want to close everything to the outside.

  • busybox is customized for the multi user setup we will do later.

    • Customize busybox options enabled.
      • Busybox Settings.
        • General Configuration.
          • Support Unicode enabled to be on the safe side.
          • Support for SUID/SGID handling needed for the su command.
      • Coreutils.
        • groups, id, chmod, chown enabled.
      • Login/Password Management Utilities.
        • Support for shadow passwords same as earlier.
        • Use internal password and group functions rather than system functions enabled.
          • Use internal shadow password functions enabled, to use busybox functions instead of the shadow package.
        • adduser, addgroup, deluser, delgroup, passwd, enabled.
        • su, enabled.
          • Enable su to write to syslog, enabled. Root access will be logged.
      • Miscellaneous Utilities.
        • crond which I think is enabled by default.
        • crontab which I think is enabled by default.

In Kernel modules I believe that everything needed is enabled by default, but there is a little more stuff that is nice.

  • Block Devices
    • kmod-loop as module. Loop devices are so neat.
  • Filesystems enable whatever you may need.
  • LED modules these might be fun.

In Languages I enable python3 and setuptools for my static site generator.

In LuCI make sure to enable the basic interface and build it as a module. LuCI is the only reliable way I have been able to flash a new image to the NAS.

In Network a lot of things like web servers hide.

  • File Transfer, I have curl, rsync and wget compiled as modules.
  • SSH enable openssh-sftp-server for SFTP access.
  • Web Serves/Proxies enable a web server, I use lighttpd.
  • Enable webalizer if you want site statistics.

Compile.

The CA-certificates package expect python to point to a python 2.x interpreter, my Gentoo system uses Python 3, which leads to missing certificates. I made a patch, that you can drop into packages/system/ca-certificates/patches in your OpenWRT directory, if you run into this.

To build everything just run make. To see all output from the build process use:

make V=s

The images end up in bin/oxnas, along with the packages. I flash openwrt-oxnas-stg212-ubifs-sysupgrade.tar using LuCI.

Adding custom packages.

OpenWRT wiki: OPKG Package Manager

I have chosen to compile most of the software I use, in this installation, as packages that must be installed after flashing the static image. Some of these packages are only installed for my own personal convenience, and some because they are needed for my static site generator.

Serving packages for OpenWRT.

Like when installing the bootstrap image you need a web server with the package files available to OpenWRT. I assume that the OpenWRT package tree is copied to the root of the server. You could copy the package files to the HDD, but I have not tried that. /etc/opkg.conf need an adjustment to tell opkg (the package manager) where to find the packages:

dest root /
dest ram /tmp
lists_dir ext /var/opkg-lists
option overlay_root /overlay
src/gz base http://serverip/packages/base
#src/gz telephony http://serverip/packages/telephony
src/gz deadbok http://serverip/packages/deadbok
src/gz packages http://serverip/packages/packages
src/gz routing http://serverip/packages/routing
src/gz luci http://serverip/packages/luci
#src/gz management http://serverip/packages/management

Replace severip with the IP address of the computer serving the packages.

Update the package index.

opkg update

Installing required packages.

File systems:

opkg install kmod-fs-ext4 swap-utils opkg e2fsprogs
modprobe ext4

Web server:

opkg install ca-certificates
opkg install lighttpd lighttpd-mod-accesslog lighttpd-mod-compress
opkg install lighttpd-mod-status lighttpd-mod-alias lighttpd-mod-access
  • lighttpd-mod-accesslog: Log access to the web server to a file.
  • lighttpd-mod-compress: Compress data before sending them to the client. +lighttpd-mod-status: Publishes some status information about the server. +lighttpd-mod-alias: Allows you to point an URL at a specific directory. +lighttpd-mod-access: Restrict access.

Installing the optional packages.

These are just tools that are nice to have.

File manager:

opkg install mc

Easy editor

opkg install nano

SFTP server:

opkg install openssh-sftp-server

USB mass storage support (aka. USB stick)

opkg install kmod-usb-storage-extras

Installing packages for ssg.

Python 3:

For some reason my package does not pull in the Python 3 dependency correctly, therefore the package python3 must be installed first.

opkg install python3
opkg install python3-setuptools

Git:

Links in /usr/libexec/git-core/ are wrong, this is corrected by creating the symlink, see Bug #11930.

opkg install git
ln -s $(which git) /usr/libexec/git-core/git

Final configuration.

System.

System configuration

Global configuration is done in /etc/config/system. I sent the logs to a file on the HDD, and limited it at 1Mb in size. You should configure the host name and time zone to your local preferences.

The log levels of different subsystems is configured in this file as well. Notice that for conloglevel and klogconloglevel a higher number means more verbose, while for cronloglevel it is the other way around.

I have not touched the time server configuration, I only use the client part, and it worked out of the box.

config system
        option hostname         OpenWRT
        option log_file         /mnt/data/log/messages
        option log_size         1024
        option log_type         file
        option timezone         Europe/Copenhagen
#Log levels 1-8
#Higher is more verbose
        option conloglevel      4
#Lower is more verbose
        option cronloglevel     4

config timeserver ntp
        list server             0.openwrt.pool.ntp.org
        list server             1.openwrt.pool.ntp.org
        list server             2.openwrt.pool.ntp.org
        list server             3.openwrt.pool.ntp.org
        option enabled          1
        option enable_server    0

Mount points.

Fstab Configuration

There are two "disks" in the system, the internal flash, and the HDD connected to the SATA port.

  • / OpenWRT on the internal flash.
  • /mnt/data Root of the connected HDD.
  • /mnt/data/www Root of the pages served by lighttpd.
  • /mnt/data/log System log files.
  • /mnt/data/tmp Temporary files.

OpenWRT uses /etc/config/fstab to configure mount points.

config global
    option  anon_swap   '0'
    option  anon_mount  '0'
    option  auto_swap   '1'
    option  auto_mount  '1'
    option  delay_root  '5'
    option  check_fs    '1'

The global section tells OpenWRT, to not mount any drives that do not have their own section in fstab (anon_). Auto_ to mount any file system and swap space, from the fstab. Delay mounting for 5 seconds, and perform a file system check if needed.

config mount
    option target       '/mnt/data'
    option fstype       'ext4'
    option options      'rw,sync'
    option enabled      '1'
    option device       '/dev/sda2'
    option enabled_fsck '1'

This section configures /dev/sda2 as an ext4 partition with read-write access, and mounts it at /mnt/data.

config swap
    option device       '/dev/sda3'
    option enabled      '1'

Last is the swap space from `/dev/sda3'.

Create the mount point and mount the partitions.

mkdir /mnt/data
block mount

Create directories for web server, logs, and temporary files.

mkdir /mnt/data/{www,log,tmp}

Adding users and groups.

OpenWRT is not build to be a multiuser system, but it is possible to configure it like that. There are two options, either use shadow like a desktop Linux system, or use busybox build in user handling. I have used the busybox version, since it is lighter.

User directories are kept on the HDD and linked into the root file system.

mkdir -p /mnt/data/home
ln -sf /mnt/data/home /home

Users are added using the adduser command. Replace username with the user name you want.

adduser username

Next create the user directory and set the permissions.

mkdir /mnt/data/home/username
chown -R username /mnt/data/home/username
chmod 700 /mnt/data/home/username

I still want root access, but I to log in as a regular user and su to the root account, like a desktop system. Busybox needs some setup for the su command to work.

chmod u+s /bin/busybox

/etc/busybox.conf

[SUID]
su = ssx root.root

Disabling root access from ssh.

Dropbear Configuration

Now that su works, there is no reason to allow root access through ssh, if you do not need ssh it would be even better to disable it.

For non root access: /etc/config/dropbear

config dropbear
    option PasswordAuth     '1'
    option RootPasswordAuth '0'
    option RootLogin        '0'
    option Port             '22'

Disable ssh enterily:

/etc/init.d/dropbear disable

File system permissions.

Permissions for /mnt/data/www/, the directory served by lighttpd.

chown http:www-data /mnt/data/www/

Configuring lighttpd.

Configuring Lighttpd, Lighttpd Secure Web Server Tutorial

Configuration is done in /etc/lighttpd/lighttpd.conf:

#Include the accesslog module to log web site access
server.modules = ( "mod_accesslog" )

#Root of the webserver is at /mnt/data/www
server.document-root        = "/mnt/data/www"

#Where uploaded files are stored    
server.upload-dirs          = ( "/mnt/data/tmp" )

#Where errors are logged
server.errorlog             = "/mnt/data/log/lighttpd/error.log"
#Process id 
server.pid-file             = "/var/run/lighttpd.pid"

#User and group that the server runs as
server.username             = "http"
server.groupname            = "www-data"

#Use index.html if root is requested
index-file.names            = ( "index.html" )
#Disable auto index directory listings
dir-listing.activate     = "disable"

#Limit request method "POST" size in kilobytes (KB)
server.max-request-size  = 1

#Disable multi range requests
server.range-requests    = "disable"

#Disable symlinks
server.follow-symlink    = "disable"

#Debug options
debug.log-file-not-found    = "enable"

#Access log module
accesslog.syslog-level      = "6"
accesslog.filename          = "/mnt/data/log/lighttpd/access.log"

#Port to bind to
server.port                 = 80

include       "/etc/lighttpd/mime.conf"
#include_shell "cat /etc/lighttpd/conf.d/*.conf"

I have disabled symlinks in this configuration, which means that the web root directory, cannot be a symlink. You will get something like 403 Forbidden if you try. The same goes for symlinks inside the web root directory, they won't work.

You can change this behavior by changing server.follow-symlink = "disable" to server.follow-symlink = "enable", but i encourage you to read this answer on Server Fault.

Enable lighttpd at boot.

/etc/init.d/lighttpd enable

I did not compile in netfilter in the cross compiled kernel I made in the previous post, so I had to recompile. Here is how to compile and install the kernel natively on the NAS.

Install mkimage.

The package u-boot-tools is needed to get the mkimage command to make an U-Boot kernel image.

emerge u-boot-tools

Compile the kernel.

Go to the sources and reconfigure them using menuconfig.

cd /usr/src/linux
make menuconfig

Compile kernel.

make zImage ox820-pogoplug-pro.dtb

Compile and install modules.

make modules
make modules_install

Create the kernel image.

cat arch/arm/boot/zImage arch/arm/boot/dts/ox820-pogoplug-pro.dtb > arch/arm/boot/zImage.fdt
scripts/mkuboot.sh -A arm -O linux -C none -T kernel -a 0x60008000 -e 0x60008000 -n 'Linux-3.11.1+' -d arch/arm/boot/zImage.fdt arch/arm/boot/uImage

Write the image to the disk.

Edit the disk_create script to change the target drive in the variable disk to /dev/sda.

Integrate the new kernel into WarheadsSE's tool.

cd /usr/src/disk_create
cp /usr/src/linux/arch/arm/boot/uImage uImages/gentoo

Write the image.

./disk_create

Done.

Happy hacking.

I bought an USB WIFI dongle, on ebay, to use with the Raspberry Pi. I thought the chip was a Ralink chip which is supported, but it turned out it was a MediaTek MT7601.

Bus 001 Device 005: ID 148f:7601 Ralink Technology, Corp.

All this is for Raspian and I have gathered all the steps needed here.

Driver

The driver is available at their web side download page here. Find the one called "MT7601U USB". I have the file mirrored here.

These instructions work for building the driver.

Become root.

sudo -s

Download latest updates.

apt-get update
apt-get upgrade
rpi-update

Download linux kernel source, this is needed to compile the driver module.

cd /usr/src
git clone https://github.com/raspberrypi/linux.git
sudo ln -s /usr/src/linux /lib/modules/`uname -r`/build
cd linux

Prepare the kernel with the current kernel config from the running system.

make mrproper
zcat /proc/config.gz > .config
cp .config .config.org
make modules_prepare

Download the module symbols of the current kernel, to avoid having to recompile the kernel.

wget https://raw.github.com/raspberrypi/firmware/master/extra/Module.symvers

Get the MT7601 USB driver into your home directory. Then, lets uncompress the file.

cd ~ 
tar -xvjpf DPO_MT7601U_LinuxSTA_3.0.0.4_20130913.tar.bz2 
cd DPO*

The default driver is really noisy and spits out a lot of debug information. This behaviour can be stopped by changing a line in os/linux/rt_linux.c from:

ULONG RTDebugLevel = RT_DEBUG_TRACE;

to:

ULONG RTDebugLevel = 0; // RT_DEBUG_TRACE;

Finally build the driver and install it.

make
make install

Raspbian configuration

Configure the ra0 interface for DHCP and make it start at boot. Edit /etc/network/interfaces to look like:

auto lo

iface lo inet loopback
iface eth0 inet dhcp

auto ra0
allow-hotplug ra0
iface ra0 inet dhcp
wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf
iface default inet dhcp

Then add your WIFI name and key to /etc/wpa_supplicant/wpa_supplicant.conf.

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1

network={
        ssid="YOURWIFINAME"
        key_mgmt=WPA-PSK
        psk="YOURPASSWORD"
}

The wireless network is then brought up by:

ifup ra0

or a reboot.

I was given a Medion MD86517 NAS without a drive for free. I wanted to put a 2.5" disk in it, and use it as a web-server. The NAS runs Linux, and the sources are here.

A large part of the installation was done on a regular Gentoo x86/x64 PC, using a SATA to USB converter. Start with a clean drive with no partitions, connected to the host computer (Not the NAS).

During the install, I have aimed to have all files needed for a new intall, located on the NAS drive itself, in the hopes that it will make a reinstall, easier. You can of course remove these files from /usr/src, if you do not want this.

Much of this stuff needs root permissions, and all the NAS side stuff is done through a serial connection. If something is unclear, read The Gentoo handbook, this is in esence the same procedure, except I boot into the system instead of chrooting.

A lot of thanks and credit to the people in this thread, without whom I would never have gotten on the right track.

Partitioning

To boot from the SATA disk, a special partition layout is needed. The ox820 reads the start of the drive, to check if it is bootable. A script has been written to put the right data in the first part of the hard disk. Download disk creation files created by WarheadsSE, extract the files somewhere, and enter that directory. Edit the disk_create script to change the target drive in the variable disk.

Creating the partitions

Prepare the disk using WarheadsSE's tool.

./disk_create

Fire up fdisk to partition the disk.

fdisk -c=dos /dev/sdb
  • Create a small partition for U-Boot, stage1, and the kernel. WarheadsSE recommends a 10M partition. This partition must start at sector 2048.

  • Create a second partition for the root file system, leave a little space left for a swap partition.

  • Create a third partition for swap space. Set it as swap type.

Format the second and third partition, I use ext4 as the root file system.

mkfs.ext4 /dev/sdb2
mkswap /dev/sdb3

Last, mount the second partition to /mnt/gentoo, your partition may have another designation than /dev/sdb.

cd /mnt
mkdir gentoo
mount /dev/sdb2 /mnt/gentoo

Root file system

Download a stage 3 Gentoo for ARM5, and extract it to /mnt/gentoo. Though the processor is ARM6 compatible, I could not get it to boot beyond the kernel using and ARM6 stage 3.

tar -xvjpf stage3-armv5tel-20140115.tar.bz2 -C /mnt/gentoo

Set the baud rate in /mnt/gentoo/etc/inittab to 115200. Change:

#s0:12345:respawn:/sbin/agetty -L 9600 ttyS0 vt100

to:

s0:12345:respawn:/sbin/agetty -L 115200 ttyS0 vt100

Copy resolv.conf from your host /etc directory, to have DNS working.

cp -L /etc/resolv.conf /mnt/gentoo/etc/resolv.conf

Create a link from net.lo to net.eth0 to enable the network at first boot.

cd /mnt/gentoo/etc/init.d
ln -sf net.lo net.eth0

Edit /mnt/gentoo/etc/fstab to set the devices for the root and swap file system. The file should contain something like this:

#/dev/BOOT              /boot           ext2            noauto,noatime  1 2
/dev/sda2               /               ext4            noatime         0 1
/dev/sda3               none            swap            sw              0 0
#/dev/cdrom             /mnt/cdrom      auto            noauto,ro       0 0
#/dev/fd0               /mnt/floppy     auto            noauto          0 0

Copy passwd and shadow from the running system to have your logins and passwords when you boot the NAS.

cp /etc/passwd /mnt/gentoo/etc/passwd
cp /etc/shadow /mnt/gentoo/etc/shadow
cp /etc/group /mnt/gentoo/etc/group

Select mirrors for portage.

mirrorselect -i -o >> /mnt/gentoo/etc/portage/make.conf
mirrorselect -i -r -o >> /mnt/gentoo/etc/portage/make.conf

Set the timezone.

echo "Europe/Copenhagen" > /mnt/gentoo/etc/timezone

Set the hostanme.

nano -w /mnt/gentoo/etc/conf.d/hostname
nano -w /mnt/gentoo/etc/hosts

Set the keymap (just in case).

nano -w /mnt/gentoo/etc/conf.d/keymaps

Last edit and change UTC to local if needed.

nano -w /etc/conf.d/hwclock

Kernel

You will need an ARM cross-compiler, Gentoo's crossdev comes in handy.

crossdev -t armv5tel-softfloat-linux-gnueabi

Clone linux-oxnas into /mnt/gentoo/usr/src.

cd /mnt/gentoo/usr/src
git clone https://github.com/kref/linux-oxnas
ln -sf linux-oxnas linux

cd linux-oxnas
make ARCH=arm ox820_defconfig CROSS_COMPILE=armv5tel-softfloat-linux-gnueabi-
make ARCH=arm menuconfig CROSS_COMPILE=armv5tel-softfloat-linux-gnueabi-

Boot options --->
[*] Use appended device tree blob to zImage (EXPERIMENTAL)
[*] Supplement the appended DTB with traditional ATAG information
disable PCI support if you device does not have one

Remember to compile in support for the root file system type, if you did like me this means enabling the ext4 file system.

File system  --->
<*> The Extended 4 (ext 4) filesystem

Compile and create kernel image.

make ARCH=arm zImage ox820-pogoplug-pro.dtb CROSS_COMPILE=armv5tel-softfloat-linux-gnueabi-

cat arch/arm/boot/zImage arch/arm/boot/dts/ox820-pogoplug-pro.dtb > arch/arm/boot/zImage.fdt

scripts/mkuboot.sh -A arm -O linux -C none -T kernel -a 0x60008000 -e 0x60008000 -n 'Linux-3.11.1+' -d arch/arm/boot/zImage.fdt arch/arm/boot/uImage

Final disk creation

Copy WarheadsSE's disk creation files (contents of onax-sata-boot.tar.gz) to the /mnt/gentoo/usr/src.

mkdir /mnt/gentoo/usr/src/disk_create
cp -Rv (Where you unpacked the files)/* /mnt/gentoo/usr/src/disk_create

Integrate the new kernel into WarheadsSE's tool.

cd /mnt/gentoo/usr/src/disk_create
cp /mnt/gentoo/usr/src/linux-oxnas/arch/arm/boot/uImage uImages/gentoo
rm uImage
ln -sf uImages/gentoo uImage
./disk_create

Preparing for first boot

Unmount and sync the disk.

cd /
umount /mnt/gentoo
sync

Remove the drive from the host computer and physically install it in the NAS.

First Boot

Set the clock. MMDDhhmmCCYY is month, date, hour, minute, century, year

date MMDDhhmmCCYY

Get the portage tree.

emerge --sync

Set the Profile.

eselect profile list

I selected default/linux/arm/13.0/armv5te.

eselect profile set 18

Configure the locales, first put the locales you want supported in locale.gen.

nano -w /etc/locale.gen

Generate the locales and select the system-wide one.

locale-gen
eselect locale list
eselect locale set *locale nr.*
env-update && source /etc/profile

Add the network interface to the startup.

rc-update add net.eth0 default

Update and install some needed stuff.

emerge -uDNv world ntp cronie syslog-ng openssh logrotate dhcpcd

Add it to the startup.

rc-update add syslog-ng default
rc-update add cronie default
rc-update add sshd default
rc-update add ntp-client default
rc-update add swclock boot
rc-update del hwclock boot

The end

You now have a basic Gentoo system running, from here you can install a web server, a DLNA server, or whatever you want.

Generated on 2015-02-28 20:54:58.952796