How to build FreeNAS from Scratch

20011 ワード

How to build FreeNAS from Scratch


Quickstart Guide for Developers and interested User
by Christian Degen aka bubulein [email protected]

Synopsis


After reading this Guide you will know how to Setup a FreeBSD System for compiling FreeNAS from Scratch. This Document will use the current FreeNAS Version based on FreeBSD 7.3. If you want to know more about FreeBSD 7.3 i refer to the FreeBSD 7.3-RELEASE Announcement
 
 
You need!
 
a dedicated PC or Virtual Mashine

a connection to the Internet

8GB Harddiskspace

FreeBSD 7.3 Installation CD ( bootonly or disc1) (see FreeBSD 7.3-RELEASE Announcement)

 
 
 
It isn't necessary to use a dedicated box to compile FreeNAS. In this Guide we will setup our FreeNAS build environment inside a VirtualBox. This virtualisation software can be downloaded for free. Get VirtualBox. Virtualbox is available for Windows, Mac, Linux and OpenSolaris. ( also for FreeBSD
Also You can use VMware or Qemu.
 
 
Note on i386 and amd64
For compiling you need to setup a FreeBSD system according to the target architecture. This means if you like to build FreeNAS for amd64 you need to setup a FreeBSD amd64 system. ( same for i386). You can install an i386 system without problems on VirtualBox running on a amd64 Host.
If you want to setup FreeBSD amd64 on an i386 Host you have to use Qemu for emulating amd64. Please follow the Qemu Manual for doing this.
 
 
Please read the documentation of your Virtualisation Software for how to setup. Following a short explanation for VirtualBox.

VirtualBox


Installing VirtualBox


Get VirtualBox http://www.virtualbox.org/wiki/Downloads

Follow the Installation instruction acording to your Operatingsystem. Manual

Create a Virtual Machine for FreeBSD


Create “New” Virtual Maschine

Selecte the Name you want to use ( FreeBSD 7.3 - FreeNAS from Scratch)

Operating System: BSD - Version: FreeBSD

Select Memory you want to assign to your VirtualBox ( 256 MB suggested)

Create New Harddrive with atleast 8 GB of Diskspace


Installing FreeBSD 7.3


Now its time to download FreeBSD 7.3 if you have not already done that. There is no better Setup Guide for FreeBSD then the original one from the FreeBSD Handbook.
see: FreeBSD Handbook - Chapter 2 Installing FreeBSD

Preparing FreeBSD


After successful installation there are some steps needed to prepare the system.In this example we install prebuild binaries Let'z Go!!

Connecting via SSH


The easiest way to work with the VirtualBox is connecting via SSH.On Linux and Mac boxes your can use the Terminal to connect via SSH. On Windows you can use a SSH-Client like
Putty which can be downloaded here: PuTTY Download Page
If you don't know the IP of your FreeBSD system type
freenas-dev# ifconfig
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 00:e0:4c:d2:7e:ba
        inet 192.168.1.250 netmask 0xffffff00 broadcast 192.168.1.255
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
[...]

Now you can connect via ssh. Use the username you created during setup and the IP of your FreeBSD/Virtual Machine. To become the superuser root type
freenas-dev# su

 
 
Why I cannot su to root?
 
Be sure that your user is member of the group “wheel”. This group is able to become superuser. If not, following command will add your user to group wheel:
freenas-dev# pw usermod YOUR_USERNAME -G wheel


If you don't have a user run
freenas-dev# adduser

Be sure to assign the user to group “wheel”.
 
 

Updating FreeBSD


The first step is to update our FreeBSD system.The command
freenas-dev# uname -a

shows you the current FreeBSD version. For example on my Virtual Maschine
FreeBSD freenas-dev.freenas.org 7.3-RELEASE FreeBSD 7.3-RELEASE #0: Sun Mar 21 06:15:01 UTC 2010     [email protected]:/usr/obj/usr/src/sys/GENERIC  i386

Start updating your FreeBSD with
freenas-dev# freebsd-update fetch install

This will fetch and install all updates.After a reboot with
freenas-dev# reboot

The command
freenas-dev# uname -a

Should show you that you updated FreeBSD. see the -p1 in my example. This means we updated to “patchlevel 1”.
freenas-dev# uname -a
FreeBSD freenas-dev.freenas.org 7.3-RELEASE-p1 FreeBSD 7.3-RELEASE-p1 #0: Wed May 26 04:29:05 UTC 2010     [email protected]:/usr/obj/usr/src/sys/GENERIC  i386

Installing necessary applications


Following the commands you should execute to install the applications we need for compiling FreeNAS from scratch.
If you like to install them prebuild ( the easiest way) use:
freenas-dev# pkg_add -r subversion / 
pkg_add -r cdrtools / 
pkg_add -r portupgrade /  
pkg_add -r php5 / 
pkg_add -r perl / 
pkg_add -r ruby / 
pkg_add -r m4 / 
pkg_add -r bison / 
pkg_add -r xproto / 
pkg_add -r nano /
pkg_add -r bash / 
pkg_add -r cvsup-without-gui 

 
 
Depending on your shell execute “rehash” to publish new commands
 
 

Fetching the ports-tree


Now we fetch the ports-tree. The ports-tree conatains all information we need to compile an application from source.
freenas-dev# portsnap fetch extract

This takes a few minutes. After fetching and extracting you find everything under/usr/ports. For more information about the ports-tree see Using the Ports Collection
If you later want to update the ports-tree you can use
freenas-dev# portsnap update

Also cvsup can be used - more on this topic later.

Downloading FreeBSD Sourcecode


For downloading the FreeBSD sources we use cvsup which we have already installed. Cvsup can also be used for updating the ports-tree. Do:
freenas-dev# cp /usr/share/examples/cvsup/ports-supfile /etc
freenas-dev# cp /usr/share/examples/cvsup/standard-supfile /etc/source-supfile

Then use your favorite editor ( nano, ee, vi) to change following lines in the files we copied above:
freenas-dev# ee /etc/source-supfile
from:
*default host=CHANGE_THIS.FreeBSD.org
to:
*default host=cvsup2.de.FreeBSD.org

see http://www.freebsd.org/doc/handbook/mirrors.html for a mirror next to you.
After editing the files run:
freenas-dev# cvsup -L 2 -g /etc/source-supfile
freenas-dev# cvsup -L 2 -g /etc/ports-supfile

for updating. ( ports-tree should be already uptodate)

Updating installed Ports


Earlier in this Guide we have installed the applications we need as prebuild binaries.Sometimes these binaries are not 100% uptodate. To avoid this you can now run
portupgrade -av

This well use your ports-tree to compile this applications again with the most actual source from the ports-tree. While updating “portupgrade” ask you some options you ould use for building the applications from source. Its ok to use the default settings.
 
 
To avoid building X11 ( Graphical Desktop related) we create or edit/etc/make.conf and add the line
WITHOUT_X11=yes
 
 
 
 
Perhaps this the right time to create a Snapshot of your system. See The manual of your Virtual Machine for a “HowTo”
For VirtualBox see: First steps - Snapshots
 
 

Building FreeNAS


Setup FreeNAS build environment


Now we are going to setup the necessary directories and download the FreeNAS SVN.Developers use Subversion to maintain current and historical versions of files such as source code, web pages, and documentation.see: Apache Subversion on Wikipedia.org
Then i will guide you step by step through the interactive buildscript.

Getting FreeNAS from svn


Everything is build under/usr/local/freenas, so we create the directory and checkout the svn code.
freenas-dev# mkdir /usr/local/freenas 
 
freenas-dev# cd /usr/local/freenas/ 

freenas-dev# svn co https://freenas.svn.sourceforge.net/svnroot/freenas/branches/0.7 svn 


 
 
 
Error validating server certificate for 'https://freenas.svn.sourceforge.net:443':
 - The certificate is not issued by a trusted authority.//
 Use the fingerprint to validate the certificate manually!
Certificate information:
 - Hostname: *.svn.sourceforge.net
 - Valid: from Mon, 04 Jan 2010 14:21:55 GMT until Sat, 05 Feb 2011 09:03:23 GMT
 - Issuer: Equifax Secure Certificate Authority, Equifax, US
 - Fingerprint: ea:d1:3e:01:cc:16:e9:9b:c2:ab:4b:0c:cc:26:5f:25:78:ea:89:b4
(R)eject, accept (t)emporarily or accept (p)ermanently?

Here you can answer with (p), cause this isn't a error. You just don't have downloaded the certificate.
 
 
 

Compiling FreeNAS


To start the FreeNAS build script run
freenas-dev# /usr/local/freenas/svn/build/make.sh

The first screen appears and we can start
Welcome to the FreeNAS build environment.
Menu:
1  - Update the sources to CURRENT
2  - Build system from scratch
10 - Create 'Embedded' (IMG) file (rawrite to CF/USB/DD)
11 - Create 'LiveCD' (ISO) file
12 - Create 'LiveCD' (ISO) file without 'Embedded' file
13 - Create 'Full' (TGZ) update file
*  - Quit
>

For a first run you should go through all steps. Following i will guide you from here to your first FreeNAS LiveCD and embedded image.
Everything should be uptodate, but we do this step again just to illustrate later builds.
Select 1 - Update the sources to CURRENT
and if everything work as expected the answer should be
Checked out revision 5235.
=> Successful

Seclect 2 - Build system from scratch
Bulding system from scratch
Menu:
1 - Update source tree and ports collection
2 - Create filesystem structure
3 - Build kernel
4 - Build world
5 - Build ports
6 - Build bootloader
7 - Add necessary libraries
8 - Modify file permissions
* - Quit
>

Select 1 - Update source tree and ports collection
          ┌──────────────── FreeNAS - Update sources ────────────────┐
          │             Please select what to update.                │
          │ ┌──────────────────────────────────────────────────────┐ │
          │ │ [X] cvsup           Update source tree               │ │
          │ │ [X] freebsd-update  Fetch and install binary updates │ │
          │ │ [X] portsnap        Update ports collection          │ │
          │ └─────v(+)─────────────────────────────────────────────┘ │
          ├──────────────────────────────────────────────────────────┤
          │                  [  OK  ]       Cancel                   │
          └──────────────────────────────────────────────────────────┘

 
 
These steps we have already done, but it doesn't hurt you if you do it again!
 
 
Select 2 - Create filesystem structure
Should explain itself!
Select 3 - Build kernel
This will build the Kernel for your FreeNAS system.
  ┌──────────────────── FreeNAS - Build/Install kernel ─────────────────────┐
  │    Please select whether you want to build or install the kernel.       │
  │ ┌─────────────────────────────────────────────────────────────────────┐ │
  │ │               [X] prebuild  Apply kernel patches                    │ │
  │ │               [X] build     Build kernel                            │ │
  │ │               [X] install   Install kernel + modules                │ │
  │ └─────────────────────────────────────────────────────────────────────┘ │
  ├─────────────────────────────────────────────────────────────────────────┤
  │                         [  OK  ]       Cancel                           │
  └─────────────────────────────────────────────────────────────────────────┘



  ┌─────────────────────── FreeNAS - Kernel patches ────────────────────────┐
  │ Select the patches you want to add. Make sure you have clean/origin     │
  │ kernel sources (via cvsup) to apply patches successful.                 │
  │ ┌─────────────────────────────────────────────────────────────────────┐ │
  │ │         [X] 3ware   3ware serial ATA RAID controller driver         │ │
  │ │         [X] a100u2  A100U2 U2W-SCSI-Controller                      │ │
  │ │         [X] aac     Adaptec Raid Controller driver                  │ │
  │ │         [X] ata     Misc. ATA driver patches                        │ │
  │ │         [X] iscsi   iSCSI initiator                                 │ │
  │ │         [X] nkpt    Patch number of kernel page tables              │ │
  │ │         [X] wol     WOL for nVidia(nfe(4)) and 3Com(xl(4))          │ │
  │ │                                                                     │ │
  │ │                                                                     │ │
  │ │                                                                     │ │
  │ │                                                                     │ │
  │ │                                                                     │ │
  │ │                                                                     │ │
  │ │                                                                     │ │
  │ └─────────────────────────────────────────────────────────────────────┘ │
  ├─────────────────────────────────────────────────────────────────────────┤
  │                         [  OK  ]       Cancel                           │
  └─────────────────────────────────────────────────────────────────────────┘


If unsure keep all settings as is. This will compile the same Kernel who is released with the official FreeNAS builds!Now your System need some time to build the Kernel.
Now you are back on the previous screen, and we start building the world
Select 4 - Building World
This will copy all necessary files from the build system the your FreeNAS environment.
Now we start building the Ports.
Select 5 - Build ports
And select “build”.Then you can select which ports you like to build. Select “ALL” for a standard FreeNAS build.
                 ┌────── FreeNAS - Build/Install Ports ──────┐
                 │ Please select whether you want to build   │
                 │ or install ports.                         │
                 │ ┌───────────────────────────────────────┐ │
                 │ │        build    Build ports           │ │
                 │ │        install  Install ports         │ │
                 │ └───────────────────────────────────────┘ │
                 ├───────────────────────────────────────────┤
                 │          [  OK  ]       Cancel            │
                 └───────────────────────────────────────────┘


  ┌──────────────────────────── FreeNAS - Ports ────────────────────────────┐
  │                 Select the ports you want to process.                   │
  │ ┌─────────────────────────────────────────────────────────────────────┐ │
  │ │[X] AA_geomraid5      GEOM RAID5 filesystem                          │ │
  │ │[X] arcconf           Adaptec SCSI RAID administration tool          │ │
  │ │[X] ataidle           Utility to set spindown timeout for ATA drives │ │
  │ │[X] bash              The GNU Project's Bourne Again SHell           │ │
  │ │[X] bsnmp-ucd         A bsnmpd module impl. parts of UCD-SNMP-MIB    │ │
  │ │[X] bsnmptools        SNMP client-side tools                         │ │
  │ │[X] cdialog           An enhanced version of 'dialog'                │ │
  │ │[X] clog              Circular log file tool                         │ │
  │ │[X] e2fsprogs         Utilities to manipulate ext2/3/4 filesystems   │ │
  │ │[X] fdisk             PC slice table maintenance utility (fixed)     │ │
  │ │[X] firefly           Multithread daapd Apple iTunes server          │ │
  │ │[X] fuppes            Free UPnP Entertainment Service                │ │
  │ │[X] fusefs-ntfs       Mount NTFS partitions and disk images          │ │
  │ │[X] geom_eli          GEOM Eli filesystem encryption                 │ │
  │ └────v(+)─────────────────────────────────────────────────────────────┘ │
  ├─────────────────────────────────────────────────────────────────────────┤
  │                         [  OK  ]       Cancel                           │
  └─────────────────────────────────────────────────────────────────────────┘


Select 5 - Build ports
Select this option again, but now choose “install” instead of “build”
  install  Install ports   

This will install all your prebuild ports along with needed files from you base FreeBSD system.
Select 6 - Build bootloader
To build and install the bootloader.
Select 7 - Add necessary libraries
Should be self explained.
Select 8 - Modify file permissions
The last step will set the correct permissions for you FreeNAS files.
Now quit with * and you are back on the previous screen.
Welcome to the FreeNAS build environment.
Menu:
1  - Update the sources to CURRENT
2  - Build system from scratch
10 - Create 'Embedded' (IMG) file (rawrite to CF/USB/DD)
11 - Create 'LiveCD' (ISO) file
12 - Create 'LiveCD' (ISO) file without 'Embedded' file
13 - Create 'Full' (TGZ) update file
*  - Quit

Now the last step in our build process. Choose one of the option from 10 - 13 to create the final FreeNAS ISO-Image or embedded file.
In this example i choose option 11 for creating a LiveCD.
Thats all
After a successful build you find your Ready to Use FreeNAS ISO and/or IMG in/usr/local/freenas
[root@harry /]# ls /usr/local/freenas
FreeNAS-i386-0.7.2.5234.checksum        builds
FreeNAS-i386-LiveCD-0.7.2.5234.iso      rootfs
FreeNAS-i386-embedded-0.7.2.5234.img    svn
bootloader                              work

Hopefully this document helps you to build your own FreeNAS from scratch.
If you have questions feel free to send me an email, or the preferred way is to ask your question on the FreeNAS forum.FreeNAS forum on Sourceforge