Technical user's guide to RACHEL
Technical user's guide to RACHEL
julianharty@gmail.com
Buy on Leanpub

Table of Contents

Commissioning RACHEL

One way to get RACHEL running on a server is to simply follow the intructions to use one of the existing image files, write that image to a memory card, and boot a Raspberry Pi using that image. This may be enough for some of us. However, that RACHEL server will be a clone of one created by developers the World Possible project team, and unlikely to be ideal for our environment and context. For instance the network settings might clash with other networks in your environment; the image may be too large, etc. We can customise RACHEL servers during and after commissioning the server.

Commissioning using image files created by World Possible

The project team provide documentation online on Google Docs on how to download, install and run RACHEL as a server on Windows computers, details are available at RACHEL-Pi Download Guide

Where to find RACHEL online

One of the most reliable locations for RACHEL images is http://rachelfriends.org/

RACHEL is also available from the same site using FTP.

What’s involved to download and get RACHEL running

Here we’ll summarise the steps needed from starting the download to running RACHEL

  • Download (which can take days, and often fails part way through, so we need ways to cope with the unreliable network and Internet connections
  • Check we’ve received what was intended (by checking a calculated code called MD5 sum).
  • [Optionally] copy the downloaded file. We need plenty of space on the local drive or partition, ideally at least 3x the size of the download.
    • When dealing with image files for Raspberry Pi’s they’re created to fit on a 64GB memory card. However, 64GB is more a marketing term than a commitment the drive actually holds 64GB of materials, they often hold slightly less. A few models of memory card are significantly smaller and the image may not fit.
    • Conversely the image is smaller than many drives so there may be unused space of several GB on some memory cards. We can ‘expand’ the image once it’s been copied to the memory card so that it can use all the available space.
  • Unpack, or write the image. RACHEL contains lots of materials, which are made up of 100,000’s of files. There are so many files that some drive formats, and some unzip programs cannot cope.
  • Update, the sofftware and contents are snapshots created at some point in the past. Updating the software and contents can increase and protect the trustworthiness of the system.
  • Protect, viruses can infect most computers, and they can be stored on the memory and computers we’re using. We can use hardware and/or software to help protect the integrity of the materials and software.
  • Use, RACHEL includes software, particularly web server(s). When these are started they can unleash the full capabilities of the materials.

Writing the RACHEL Pi image to memory cards

On Microsoft Windows, there’s an excellent simple utility called Win32DiskImager which is available from http://sourceforge.net/projects/win32diskimager/files/latest/download Details of how to use this command are in the above online document.

On Linux we use the dd command to write the downloaded image to a memory card. A very clear guide is available online Installing Operating System Images on Linux

Here’s an example of writing the image where I’m using the time command to measure the duration of the write.

time sudo dd bs=1024k if=rachelpi_64EN_3.2.1.img of=/dev/mmcblk0

You probably need to revise the command line to match:

  • the input file (the if= parameter).
  • the output ‘file’ (the of= parameter) for where your computer has mapped the physical SD-card in it’s list of devices.

The write took 86 minutes! 

1 real	85m33.743s
2 user	0m0.383s
3 sys	2m2.943s
4 
5 62864228352 bytes (63 GB) copied, 5133.74 s, 12.2 MB/s

You may also need to stop your computer from using the SD-card by un mounting it, using the umount command in Linux.

For OSX there are diskutil commands. For now, please search online for instructions on how to write Raspberry Pi images when using OSX.

Configuring an existing RACHEL server

TBC for now please see the following links (from ka-lite) which cover many of the same topics

Here are the topics I want to cover in this chapter

  • network
  • server name
  • content packages
  • software packages
  • Configuring accounts in ka-lite

Introduction to the Technical User’s Guide to RACHEL

Aims of this book

Different flavours of RACHEL

Android, Windows, Debian (on Raspberry Pi’s)

Proposed Agenda

Today’s 1-day workshop will help you to learn how to use free, open software and materials to help improve education wherever you work. We will work together for several sessions so we can learn with and from each other. We have various topics to try and cover during today, our progress will depend on you and your needs, questions, and involvement. My main role is as facilitator and coach based on my work in applying digital education in various countries, both directly and by helping other people. I hope we will manage to establish strong working relationships today and during this elearning-africa conference.

What is RACHEL?

RACHEL (Remote Areas Community Hotspots for Education and Learning) is a World Possible initiative to leverage the large volume of open and free educational courseware and libraries available online and make them available to Emerging Countries schools and communities with no Internet access or very limited bandwidth. The RACHEL Initiative has made possible that offline educational servers could be deployed by World Possible and large number of NGOs all around the world.

Why would people need a “RACHEL-Pi” (RACHEL Server on a Pi)?

  • A large amount of rich educational content and open courseware is available online, in thousands of web sites, and are available for free to anyone, anywhere; as long as you have an active Internet connection.
  • Unfortunately in the world there are still large number of regions and communities with no Internet access at all, or in many other cases if they have Internet connectivity the available bandwidth is unusable for most practical purposes, much less to watch videos or rich media lectures or textbooks.
  • Or Internet connectivity is too expensive.
  • Or a school or community has good Internet connectivity but for some reason access to many web sites is blocked, like YouTube, where video lectures of multiple colleges and organizations are hosted.
  • This Raspberry Pi hardware and this RACHEL-Pi server distribution provides a very affordable, very compact, low-power server platform, which is very convenient to transport, ship, setup and maintain on on-the-ground deployments.

What type of content is on a “RACHEL-Pi” distribution?

  • An offline self-contained very comprehensive and curated Wikipedia Encyclopedia.
  • Thousands of Khan Academy video lectures on Math and Science from khanacademy.org .
  • More than 3,000 eBooks, organized by bookshelves, from Project Gutenberg.
  • Full Medical Encyclopedia, MedlinePlus, from the U.S. National Library of Medicine and the National Institutes of Health.
  • The latest Medical publications and Health Guides from Hesperian.
  • The OLPC Collections form the OLPC project.
  • Teachers resources from the UNESCO International Institute for Capacity Building in Africa
  • And much more…

An overview of RACHEL: the software and the contents.

RACHEL is a project originally created not in the technology hotspots of Silicon Valley, London, or the First World; instead it started in Central America where volunteers wanted to help improve education in their schools, even when there was little money to pay for materials, books, or even teachers. The project expanded from Spanish to English and several other languages. As we were able to collect and collate more materials specific versions became possible, for farmers, for medicine, etc. and in Kenya, for instance, demand for installation exceeds the ability of the small World Possible team.

RACHEL comprises several free software applications and lots of content that’s free to use and free to distribute. Technically there are several technical flavours, these include:

  • RACHEL for Windows computers (RACHEL-on-a-stick)
  • RACHEL for Android
  • RACHEL for Linux computers, including Raspberry Pi’s, Ubuntu, and other Debian-based systems.

The project continues to improve and evolve. Currently releases are packaged as single massive ‘image’ files. The main distributions are packaged for USB memory, typically on a USB memory stick, or on an sd-card for either Android or Raspberry Pi computers. Technical users can tweak these distributions so they can work on Android devices, other operating systems, etc.

Blocks of software

Blocks of Software in RACHEL

Blocks of Software in RACHEL

  • explain the main block diagrams;
  • perhaps explain why we have various servers, etc. and the opportunities to improve performance, reduce complexity, etc.

Power

Computers only run when they have sufficient good-quality power. Many portable computers include a reserve of power in a battery. How long the computer can run on a battery varies on factors such as power management software, power consumption, and the condition - or health - of the battery. Small computers, including Raspberry Pi’s can run effectively even using realtively small external batteries for up to a day. We can configure the software to use less power and thereby enable the computer to run for longer. Some batteries, including those in Laptops and Android Tablets can be charged while they are powering the rest of the computer. Some models of external battery can do likewise, and effectively behave as uninterruptable power supplies (UPS) for Raspberry Pi’s, etc. This means we can use these computers even when power is problematic, or if we have lost power for hours at a time PROVIDED we can find an effective power source to recharge the battery before it has sacrificed all it’s capacity (keeping the computer running).

In some rural areas there is little or no practical power on site, and even some areas in cities lack reliable mains power. Sometimes we can charge devices (and external batteries) nearby, for instance in an Internet cafe, or at home. Sometimes we can use suitable solar panel(s). Ideally batteries and devices should be powered while they are being used so the don’t wear out the internal chemistry of the batteries. If we have several batteries, we may be able to charge some when they are idle, or where it is impractical for the source of power to reach where the device needs to be used.

How to keep the computers running even when power is faulty or not available. Batteries, Charging, Solar Panels, Power Adapters, etc.

Good-enough quality of equipment, including chargers

There is a vast range of equipment available for virtaully every aspect of technology (apart from Raspberry Pi’s where the choices are limited to which model to use). The qualities of equipment varies massively. Some equipment is of very poor quality and has a high risk of failing or causing connected equipment to fail. Some may even electrocute us. Others may provide high-quality service for decades. Genuine & branded equipment tends to be more reliable than unbranded items or counterfeights. We can and should test new equipment. Power meters can help to show the power characteristics of the devices.

  • power meters: useful to measure the voltage, current and watts generated or used by the other equipment being used.

Network : Wi-Fi, 3G, Connections and how to connect to existing computer networks or create a new network. Understanding the mysteries of TCP/IP. Living in a fallen world: coping with viruses on your computers and existing computers. Working with and controlling the Operating System: All computers have an Operating System, a good understanding can help us control the computer, diagnose and fix problems, and install new software. Content (Educational Materials): Discover how contents are structured on the Raspberry Pi, how you can add additional materials so that people can use it.
Apps: discover and learn how to install software apps on Android Tablets and software on Raspberry Pi’s. Keeping Current: Content and software ages and may become less relevant. Learn various ways content can be updated. Wrap-up, Q&A and next steps.

Safe Working Practices

  • protect your equipment against viruses
    • write protect tabs, read-only storage e.g. CD-ROMs, DVDs.
    • use trustworthy anti-virus scanners, real-time protection, etc.
  • protect your equipment against adverse physical conditions (dust, water, salt, heat, etc)

Clean the computer before you work

“An ounce of prevention is worth a pound of cure.” On the assumption that Windows computers in the field might have viruses, let’s scan it for potential viruses before plugging in any of our drives. Mirosoft Safety Scanner is one of several useful software tools which they update frequently to try and catch even very recent viruses. It needs to be downloaded while you have an Internet connection, it can then be used offline. They provide two versions, one each for 32-bit and 64-bit Windows installations. If practical, create and use it on a DVD-ROM to further reduce the risk of infecting your USB drives.

Use read-only options where practical

Read Only Memory (ROM) is written once, then not updated again. We’re familiar with CD-ROMs and DVD-ROMs. Where practical using these reduces the chances of getting computer viruses to virtually nil.

SD-Cards can have a ‘read-only’ swtich. Provided the computer honours this switch, we can use it to protect the contents of the drive. Micro-SD-Cards and USB drives don’t have this protection.

Sadly there are few cost-effective ways to switch USB or Hard drives to read-only. There are some relatively expensive connectors, from around $500 USD, for those who can afford them. Examples of these devices include: * Tableau T8-R2 Forensic USB Bridge

Advanced topic: Configuring Windows Registry

Windows XP apparently includes a registry setting to treat a drive as read-only. Using RegEditcCreate a Registry key of HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\StorageDevicePolicies then create a DWORD called WriteProtectand give it a value of 1. See the more complete instructions at Configure USB Disks to be Read-Only in Windows XP SP2

Note: these instructions haven’t been tested and may not stop powerful viruses which may bypass this somehow.

Leave the computer as you’d wish to find it

There are plenty of choices of anti-virus software. They vary significantly in performance, effectiveness, behaviours and price. We’re not aware of a ‘best’ choice. If the computer has Windows 7 or later then the Microsoft Security Essentials software is a reasonable option. For Windows XP, we need another product (or potentially we could switch to Ubuntu as the operating system?)

We recommend checking reviews online as anti-virus software and products change rapidly.

Coping with adverse environmental conditions

My first visit to Dadaab refugee camp, a semi-desert area with 40C air temperature, was enough to remind me how sensitive some computers are to physical conditions. Computers designed for air-conditioned offices in the first world may suffer and fail more rapidly in harsh conditions. Where you have choices on the equipment used, consider stuff that is resilient and robust against whatever conditions we might expect or predict based on where the equipment will be used. Paradoxically, some of the most inexpensive equipment may cope best.

We may also be able to protect the equipment, and also maintain it, to reduce the risk of it failing when it needs to be used.

Powering your equipment

Computers need reliable power. Reliable power is especially important for the computers used to host RACHEL because these computers need to supply and collect information from many users at the same time. If the power fails, or just dips or spikes while information is being collected and before it is safely saved, the information being collected can be corrupted losing that information and even making it difficult or impossible to restart RACHEL or reload the database with the information when the power is good.

All computers, especially computers that do not have internal batteries can have their files corrupted by bad power. File corruption caused by bad power are more common a RACHEL Server is collecting information such as the answers supplied by students for question is some of the teaching material such a the Khan Academy math lessons in KA-Lite.

Bad power or power failure can cause problems even with the parts of RACHEL that only supply information and to do not collect or change the information on the hard drive or SD Card. Even computers that are used by just one person at a time can have their files corrupted by bad power. The reason for this is that most computer operating systems open files while they are working and need to close these files when they stop working and erase small files were made only for temporary use. See these sections on shutting down RACHEL for examples of what is required before disconnecting the power from a computer acting as a RACHEL server.

Some computers, especially laptop computer, tablets, and smartphones, have their own battery to supply power when they are disconnected from external power supplies. This internal battery power makes laptops and tablets good for traveling and for areas with unreliable power. Laptops and powerful tablets can be a good computer for a RACHEL server in areas with unreliable power because they can be programmed shut down properly when the battery is about to run out of power.

In the spring of 2015 World Possible started selling a RACHEL Pi server with a built-in battery and a control board for areas with unreliable power: The RACHEL Pi 2 Remote Deployment Pack. See the World Possible Store at: http://world-possible.myshopify.com/ There is only a small battery in the RACHEL Pi 2 Remote Deployment Pack. This small battery can only supply enough power for about 10 minutes of operation. The system then shuts down automatically in a careful process that takes about 3 minutes. Therefore, this server should have an external source of power that only has only a few outages that are longer than 10 minutes. If power outages are going to be longer than 10 minutes and large car battery or solar battery charged with solar panels or wtih AC power from the electric company or a generator should be added, not to protect the server, but to provide reliable service to users. Of course, if all of the users are dependent upon AC power from the electric company or the power company to run their computers, then there is not benefit of have a large battery for the server. However, many users now often use laptops or tablets so having a server with a reliable power source will enable them to keep working.

With poor quality that could potentially cause damage to the contents of the computer or even the computer itself it is possible to filter, or condition, the power so the quality is improved. This conditioning is especially good for protecting the computer hardware and also for protecting the software from short dips or surges.

An external battery can be added to most computers to enable them to have a reservoir of power to protect both the software and the hardware. To be sure to protect the software the computer needs to be shutdown before the power in this reservoir is used up. If the power to a computer dies or is turned off before it is shutdown, most of the times the problems with the files will be so small, the computer will start without any visible problems when it is turned on. However, just because this appears to be OK, the problems when they occur can be very difficult to correct, especially if back up copies of the software for the operating system are not available.

Often the external battery is combined with power conditioning which can help increase the reliability of the computer.

When the computer consumes power, that power leaves the battery. Eventually the battery will run out of power. When the battery runs out of power it can no longer power the computer. If nothing else happens the computer will stop, sometimes abruptly. It’s wise and prudent to shutdown a computer before all the power has been used up. Some computers are programmed to do this automatically, others may need us to tell them to shutdown by using commands. Commands are covered in another chapter of this book.

Some batteries can be recharged while they are providing power, others are limited to either being charged or providing power. Generally we prefer using batteries that can be recharged while being used, otherwise our computer will not be available when the battery is being charged.

Power for low cost computers and Android Tablets

Most of our projects use computers powered using USB connections. USB power is 5 volts DC. Therefore much of our work and equipment focuses on USB power. This includes the solar panels, batteries, and power adapters/chargers.

USB power

USB was originally designed to enable computers to interact with peripheral devices, including keyboards, mice, network adapters, and storage. Many of these devices needed power to be provided, so the USB standard allowed these devices to use up to 500 milli-Amps (mA) of power at 5 volts, a total of 2.5 watts (watts are a common measure of power).

As devices became more sophisticated some of them needed more power than the original USB standard permitted. Some manufacturers devised their own custom solutions, eventually the industry established standards that allowed devices to communicate with the provider of USB power to negotiate more power for that device.

The devices and the power provider both need to use a common protocol to communicate effectively. Otherwise the power provider should limit the power to protect itself and the device that is asking for power. Miscommunications between the device and the power can cause problems for us, devices may not work properly when they don’t have enough power. Poor quality power supplies may overheat, short-circuit or fail.

How batteries work

Batteries use chemical reactions to store power and to provide power. The chemical reactions generate heat, and too much heat may be dangerous to us and to the battery. Over time and as batteries are used they lose their potency. Eventually a battery may become so work out that we cannot usefully use it any more. Old batteries can be recycled which protects our environment.

Batteries have a defined capacity, measured in milli-Amp hours (mAh). They may have a limit on how much power they can provide at any one time. This may affect us if our computer(s) need more power that the battery is able to safely provide, so we try to design our systems so the battery is able to provide more power than whatever is connected to it. The actual capacity of a battery decreases as it loses its potency. Often the decrease also affects how the power is delivered, where a battery suddenly becomes exhausted. We may need to account and allow for this behaviour when designing and implementing a e-learning system, otherwise a system that initially works and serves the users well may start to fail, perhaps in a few months time. Once a system starts failing and becoming unreliable it’s much less useful for the users who may become frustrated and stop using the system.

Continuous batteries vs. offline while charging

Some USB battery packs are able to be charged while they continue to provide power to devices connected to them. Other batteries stop providing power while they’re being recharged. For most situations the difference is not important, however when the battery is used to power a Raspberry Pi (or other similar single board computer) the difference has a major effect. With the continuous batteries the computer continues to run regardless of whether the battery is being charged or not (unless the battery becomes exhausted i.e. it’s been fully dischaged). With the other batteries, the computer stops as it has no power when the battery is being charged.

If the offline-charging batteries have enough power to support whatever is connected for as long as we need that computer to run, then we may have an overall system that’s acceptable - provided the computer is shutdown safely (to protect the contents of the system on that computer). We could potentially devise an overall system where we have several batteries, one in use and the others either charged or being charged. Then the computer is shutdown, for example at the end of the school day, and the battery replaced with one that’s been fully charged, and the previous battery can then be charged offline.

I don’t know why some batteries work as online batteries, and others only as offline batteries. Perhaps the online models have more sophisticated charging electronic circuits.

We can test and measure the behaviours of various batteries in order to buy the most suitable and available models, and so we can design viable e-learning systems. We have found that the behaviours of batteries vary significantly. Some models from the same brand may work as online-batteries, but others work as offline batteries.

How to test whether batteries support online charging

All batteries appear to support offline charging (otherwise they’re unlikely to be effective as batteries!). Only some work as online batteries. A simple way to test is to connect something that draws significant power, at least 500 milli-watts, which indicates that it’s working - for instance a bright LED light - then connect power to the battery. If the light dims, flickers, or extinguishes then the battery is not suitable for online use. This test isn’t quite sufficient as computers may be more sensitive to power problems, so batteries that pass the simple test can then be tested again with a Raspberry Pi connected. Connect something to the computer (either a display or a network connection) and make sure the computer is running OK. Then try a similar test, connect and then disconnect the charging power to the battery. If the computer continues running than we have a battery that may work acceptably in online mode. We’ll only really know whether it’s acceptable through longer term testing or use (which is a form of testing in practice).

Charging and recharging

Batteries are designed and specified for a number of recharge ‘cycles’; for instance I recently bought 2 USB batteries who promise 500 recharge cycles - they should continue working for at least 500 discharges and recharges. Where practical we may want to design our systems so we reduce the number of charge cycles, some computers are designed to do this automatically, however for Raspberry Pi’s, recharging USB batteries, and other low-cost small computers we may need to devise a system that does this instead.

There are several ways we can design these systems. For instance we could have several batteries where we use one until it is almost fully discharged, then unplugged from whatever it’s powering and then recharged fully, perhaps using a solar panel or a power adapter/charger.

Solar Panels

In the last few years solar panels have become available for USB power. The panels seldom meet the claims in terms of how much power they provide, and the power may not be suitable to power a computer directly, so we need to test the behavour of the panels and design systems so they provide suitable ‘clean’ power for the computers.

The power output varies based on the position of the panel relative to the sun. If we install the panel, for instance on a roof, we need to consider and test various locations and angles of the panel so that we can harness as much of the solar power as practical. Measurement tools can help us to do so.

Power adapters/chargers

There are various words used to describe the USB power adapters or chargers. In essence these devices consume mains voltage power (often around 230 volts AC) and convert that power to 5 volts DC suitable for USB-powered devices such as Android Tablets and small computers such as the Raspberry Pi.

Each charger has particular characteristics in terms of the quality and amount of power they provide, particularly when the quality of the mains voltage varies, such as during overloading of the power grid, brown-outs, etc. It’s unwise to assume that we can use any or every power adapter for our projects, poor quality chargers may cause problems for the services provided and even cause the computers to fail. In some countries there are lots of fake branded equipment, including USB chargers that claim to be from top brand companies such as Apple or Samsung. We need, therefore to choose wisely. Testing the bahaviour of chargers may help us decide which ones are going to work well for our project, context and situation.

Measuring power

Low cost power meters are available for USB. We can use these meters to measure power flowing through a USB cable. This means we can measure power provided from a solar panel, battery, or USB charger. DC power is measured in Volts and (milli-) Amps. The meters may not be 100% accurate, however once we have a good idea of how inaccurate a meter is we can use it to provide representative or comparative readings. I travel with at least one USB power meter so I can easily measure the power at various points in a system. Be careful as the power meter often needs to be connected ‘inline’, between the source and the consumer of the power. Connecting and disconnecting the power meter disrupts the power to the consumer and if that’s a computer it may stop working, and potentially fail.

Power engineering

Power engineering enables us to understand how devices use power, the impacts power use has on the availability and viability of the system, and ways to implement more efficient and effective systems.

Duty cycles

Duty cycles measure or assess how long a device can be in-use compared to when it needs to be idle. Some things are always in use, until they expire. Others are used for a period then idle (or turned off). For our purposes duty cycles help us assess how long particular computers or devices can be used uninterrupted, without needing to be retired or turned off.

For Android Tablets, if we can’t charge them while they are being used, then the duty cycle will be quite low, as often tablets take longer to charge than how long they can be used without interruption. For instance, the Google Nexus 7 can be used in a classroom on battery power for several hours, before it then needs to be recharged. The charging often happens away from the classroom, perhaps in a separate office, and may take the rest of the day to complete (especially if there are many devices and few sources of charging).

  • One way to siginficantly increase the duty cycle of the tablets is to provide charging facilities to the device, for instance using USB chargers in the classroom.

For Raspberry Pi computers, if they rely on using a battery then their duty depends on whether the battery can be charged while it’s being used to power the Pi. If so, the duty cycle can be extended significantly, potentially to many months if the power is sufficiently available. (Outages of say an hour or so wouldn’t have a significant event provided the battery has enough capacity to keep the device running for at least that period.)

Power consumption

The more power a device uses, the more it needs - that should be obvious. Therefore it’s worth spending some time seeking and selecting devices that use less power, provided they meet other criteria such as quality, price, etc.

Sometimes we can reduce the power consumption, for instance by reducing the brightness of the screen of an Android tablet in the Settings application. When we are able to reduce the power consumption, the device can be used for longer and may therefore be more useful in terms of learning and teaching, etc.

With Raspberry Pi’s the main factor is likely to be the choice of model, for instance the Model A’s use significantly less power than the Model B’s - however the reduction comes at a price in terms of performance and capabilities. It’s also possible to tweak the configuration of the software to reduce power, however it’s a relatively specialist topic outside the scope of this book currently.

Efficiency

When a computer is on it uses power, as you probably well know. Some of the power generates heat. That heat is not useful (unless you’re cold) instead it means we need more power to keep the computer running. For very limited power sources such as small solar panels, the power loss may have a significant impact on being able to keep devices charged and working. There’s little we can do practically, apart from test equipment, including USB chargers and batteries to determine how efficient they are. Generally we prefer devices that are more efficient.

“Robbing Peter to pay Paul (or compound losses)”

We may decide to charge a battery from another battery, for instance an Android tablet from a USB power pack. When we do so, power is lost (or used) twice in the process, firstly when the power pack is charged, and secondly when it charges the Android tablet. This may be necessary in some circumstances, however as a general practice, it may be worth trying to charge each battery directly. Exceptions include:

  • Charging an Android tablet or phone from a solar panel. Some devices can cope with the voltage variations, others may fail either temporarily (needing a power reset to recover) or permanently if some of the internal hardware fails. We risk the cost of destorying a device to find out whether it will cope, again sometimes that may be a practical, albeit unattractive, choice.
  • Where there is no source of power locally, for instance the batteries may be charged in a local town and needed to charge tablets in a rural school.
TBC
  • Extended use, overheating, etc.
  • Extra capacity in the system to cope with peaks in demand.
  • Power engineering when power is intermittently available e.g. Solar power doesn’t generate useful power during the night.

Using Android

Android is probably the most popular operating system, or platform as I’ll describe it in this book, in the world, at least in terms of the number of users and devices. For RACHEL we’ll focus on Android tablets, as they can be used both individually and by small groups. Thankfully, smartphones running Android behave similarly and can be used interchangably. However, the screens of smartphones tend to be much smaller and so they are less suitable for groups.

There are new versions of Android several times a year. Sometimes the changes include a major release upgrade. Sadly, few existing Android devices receive the major upgrades so we need to make sure our software works well on both the latest and older, still popular versions. At the time of writing Android 2.3 is still used by many users so ideally our software will work with Android 2.3 and later releases.

RACHEL and Android

Android devices can be used in at least 3 ways for RACHEL (and related software).

1 As a Web browser to view and interact with the various web-based materials available from a RACHEL server. 2 To run a basic version of RACHEL on the Android device. This can be shared over Wi-Fi with others. 3 To run some of the software so people can use it directly on the Android device. Examples include Kiwix (the Wikipedia offline reader) and ka-lite (mathematics exercises).

Software programs need to meet various security requirements to run on the Android device. Most Android apps (applications) are downloaded from Google Play where the apps are ‘digitally signed’ before being sent to the Android device. The Android device checks and approves the signature before allowing the app to be installed. Some of the apps we use are available from Google Play, for instance kiwix is. Others are still experimental or don’t meet the criteria Google requires, for instance they don’t approve of Web Server apps for security reasons. This means that we provide apps that haven’t been signed by Google Play. Normally Android devices refuse to install these apps to protect the device. However, it’s possible to change the security settings on the Android device to allow these ‘unsigned apps’ to be installed.

Changing the security settings for unsigned apps

Security settings are changed in the inbuilt Settings app. The details vary by device and version of Android. We will add examples for various devices in future versions of this chapter.

Choosing Android Tablets

There are lots of variations in various Android tablets available globally, and even in particular countries there may be 10’s or 100’s of different models available at a wide range of prices. Here are suggestions and recommendations on how to choose suitable devices.

Firstly here are some selection criteria:

  • Durability<⇒Fragility: durable devices will tend to last a long time in good working condition. Fragile devices may fail pysically e.g. the screen may be smashed and stop working, or internally where the device may fail to boot or run. Good quality covers and screen protectors can help protect the device and increase the useful life of the device.
  • Available Storage: RACHEL, Kiwix, and ka-lite include large volumes of materials. Some tablets have very limited storage available e.g. 1GB, which would limit it to smaller Wikipedia databases. Others may have as much as 50GB available, enough for a full installation of RACHEL (currently roughly 30GB) and other materials. Many devices include a micro SD-card slot which means the storage can be expanded using suitable[1] micro SD-cards. Many devices claim they support a maximum of 32GB micro-SD cards, for a full RACHEL installation 64GB is more practical.
  • Screen Quality and Resolution: Higher-end tablets tend to have far more legible screens that are also easier to read in various lighting conditions.
  • Physical dimensions and weight: larger screens tend to be easier to read, particularly by a group of people. However, they may be less portable. Lighter devices are considered better, particularly if younger children will be using them as they lack the strength of older children and adults.
  • Power conspumtion, battery duration, recharging:
  • Theft, resale values, traceability:
  • CPU, RAM:
  • Sensors, camera(s):
  • Audio quality:
  • Connectivity options: USB, Wi-Fi, 3G, Bluetooth, etc.
  • Operating system version:
  • Brand, support from manufacturer:

Useful life

What do we intend to use them for?

Our budget

Context

When selecting devices it’s also worth considering the contexts of use of these devices.

  • Availability of usable electricity.
  • Quality and availability of Internet access.
  • Sharing the devices, per-user accounts, etc.
  • Availability of useful, relevant contents, formats, apps, etc.
  • Physical environment: dust, heat, sand, liquids.
  • Age of primary users:

Software needed to administer Android devices

  • adb (needed on all operating systems)
  • Windows Device Drivers
  • OSX Android File Transfer

adb (Android Debug Bridge) provides most of the functionality needed to administer, debug, update and maintain Android devices. Generally we need to instal the software tools, unless someone else has already installed them on the computer we’re using. The Android device also needs to be configured to allow the adb software to interact with it. We’ll cover what to do shortly in Enabling Debugging.

Enabling Debugging

Installing apps

Using adb provides an integrated way to send and install and Android APK. Otherwise Android generally allows apks to be installed locally providing various conditions are met.

  • using email
  • using a web page
  • using file transfer & opening the apk file on the device ###Uninstalling apps Apps can be uninstalled from the Settings application. They can also be deinstalled from another computer by using adb and the root Java package name e.g. adb uninstall org.kiwix.kiwixmobile ;

Formatting SD cards

  • TBC Explain or link to infomation on disk formats.
  • Requirements for RACHEL and large ZIM files constrain the options.

Google & other Accounts

Many devices, including those with Google Apps, require a valid account configuring on the device. Some e.g. Nooks need their account as part of the initial configuration - the device cannot be used until it has a valid account that’s been verified by their servers on the Internet. Of course, this means we need a working Internet connection as well as whatever details the account-provider demands. For some countries in Africa setting up an account may be challenging, especially if they want a UK or USA address and credit card details for that account holder.

Examples of various accounts

  • Google:
  • HTC:
  • Nook:
  • Amazon: For Kindle Fires.

Custom Android Operating Systems

Cyogenmod is the most popular and comprehensive creator and provider of alternative Android releases for many devices. For popular devices there may be several versions already available from this project. These alternative releases may mean we don’t need Wi-Fi to configure the device, similarly these custom versions many not need any of the accounts that the standard platform does.

Using Microsoft Windows

Microsoft Windows, collectively is the most common operating system used on laptops and desktop computers. It has been available as various releases for several decades. The most prevalent version we’ve found on site is Windows XP which is a very old and dated version no longer supported by Microsoft. However, since it’s still popular it’s important to cover it to help you work effectively with RACHEL and related projects, applications, and materials.

We’re only going to cover various topics we consider essential, and there’s too much to say to cover the topics comprehensively here. Please be willing to investigate and explore other sources of information.

Topics to cover

  • Viruses: how to detect, remove and prevent viruses on Windows computers
  • Administrator Access
  • Terminal Emulator (putty)
  • File copying to external USB drives and connected devices such as Android tablets.
  • File Transfer over networks
  • Networking
  • Mini web servers
  • 7-zip, not any other unzip software on Windows. Some have restrictions, others viruses, and other stuff we don’t need or want. The download is http://www.7-zip.org/download.html

The murky world of Viruses

Pirate copies of Windows

Microsoft wants and needs to earn money to survive and thrive as a company. It licenses versions of Windows to earn money. In return they provide free updates and other software utilities for many years for each release of Windows. They designed Windows so that installation has a unique license key, to identify valid installations. However, people, being inventive, and perhaps unwilling to pay for a license to use Windows have been using pirate copies of Microsoft Windows. Microsoft also uses software to detect whether the installation of Windows is licensed and genuine. If so, they provide updates which include improvements to the security, and they provide anti-virus and other related software tools to try to reduce and prevent viruses from adversely affecting the computer. Pirate versions of Windows are less likely to have the security updates and they may be more likely to be at risk of viruses and the attendant problems.

We advise you to only use genuine licensed copies of Microsoft Windows if you want to use Windows on oyur computer. That way you can take advantage of the latest security updates and other improvements. And for Windows 7 and later, Microsoft provide good quality free anti-virus software: Microsoft Security Essentials. ###Administrator Access

  • Run As Administrator right-click menu

Anti-Virus software blocking RACHEL

  • stops some services from being available at all
  • we can often fix the problem by adding explicit exceptions ###Terminal Emulators
  • Putty: Free, basic useful software that enables a terminal window for remote, connected devices.
  • Cygwin: A comprehensive environment that provides an Unix/Linux-like working terminal including many of the utilities and commands administrators find useful.

Networking

  • ipconfig: a terminal command used to know the current IP address, and to request a DHCP address if the computer doesn’t obtain one automatically.
  • ping: tests connections to computers. Those computers need to accept and respond to ping requests, some are configured to ignore them for security reasons.
  • Contol Panel, Network settings: a GUI to configure the various IP network parameters.

Pitfalls

  • Commands refuse to execute e.g. Win32DiskImager.exe - Some programs, particularly for disk utilities, need to be run using an Administrator’s account. You will need to know the details of an Administrator’s account - the username and password. Once you have the details you can run programs, including Win32DiskImager by right-clicking on it and selecting the Run as Administrator option from the context menu.

Offline Installations and Offline Working

Many programs, such as Google Chrome, Firefox, Flash Player, etc. rely on a fast, reliable network to install themselves when run in the default mode. However, the internet connection may be slow, unreliable and/or expensive so we’d like to minimise the need for the installation process to use the Internet each time (e.g. for each computer if we have several to set up). Thankfully, each of these programs offers an offline installation option.

Alternatives to Windows

If you are prepared to invest significant time learning how to use another computer operating system, we recommend installing the Ubuntu operating system. This has much in common with the software used on the RACHEL Raspberry Pi server so the skills you learn can be used fruitfully on both computers. Ubuntu has several flavours and releases. We suggest you use an LTS version of Desktop Ubuntu.

We have not included details of how to download, install, confiure or use Ubuntu here. There are plenty of materials available online. Perhaps we will be able to include some of the materials either on USB memory or as part of the materials on future releases of RACHEL server.

Linux

Linux is installed on billions of computers, including Android, laptops, Raspberry Pi’s and large servers hosting much of the Internet. It has a long pedigree and has proven to be extremely flexible, adaptable and capable. However many end users have gotten used to Microsoft Windows and Microsoft Office and for many people their first impression is that Linux seems scary, hard work, and unforgiving. RACHEL is available for Windows and for Android and you don’t need to know Linux for either of these distributions of RACHEL. However, RACHEL is more capable when used on Linux, and for technical users, once they learn how to work with Linux they discover they can do much more than they’d otherwise be able to do using Windows.

Why do we find Linux useful

Working in Linux is very consistent once we’ve learned how to do so. It has tremendous flexibility and is well suited to running web servers, etc. as we use for RACHEL. Much of the software is opensource and therefore offers greater opprtunities to modify and enhance it so we can customise the software to suit our needs.

How to obtain Linux and prepare to install it

For the purpose of RACHEL we use two distributions of Linux:

  • Raspbian: on the Raspberry Pi. Available from www.raspberrypi.org
  • Ubuntu (LTS): LTS indicates this version of Linux will be supported for a long time, several years. This means we can install and use it knowing it will still be supported for the duration of most or our projects.(and it can be upgraded on an annual basis to the next release of Ubuntu LTS so we can keep going indefinitely). Downloads are availale from www.ubuntu.com/downloads

Installation options

There are lots of choices for where and how we can install Linux. For desktop versions (Ubuntu in our case) these include: USB drive / CD-ROM, VirtualBox, Dual-boot, Vagrant, etc. For Raspberry Pi’s we create the image using software utilities where the software is written to a SD-card.

TBC

Useful sizes of partitions for working with RACHEL

RACHEL images currently take up lots of space, around 64GB for the larger Raspberry Pi image, and 30GB for the USB image. This means we’d be wise to have a working partition of at least 2x the size of these two combined files so we can download an updated version without running out of space, etc. Often it’s worth also having a large capacity external drive (also known as a ‘HDD’ - which is short for Hard Disk Drive) with at least 500GB available to backup images, and enable the USB RACHEL image to be unzipped.

So, we recommend a partition where you have 200GB available to work with RACHEL images. This holds whether you’re using Windows or Linux on your computer.

Security and other updates

All software becomes obscelete eventually. The software may also have security vulnerabilities that others discover how to exploit. Thankfully, many projects provide updates to their software which are freely available to use. We simply need to install them. For some apps e.g. ka-lite we may need to run commands to backup, and update databases. We cover how to update Linux software later in this chapter. Updating apps (including ka-lite) is a separate topic TBC.

Common development tools and commands

  • Text editors: nano is probably the simplest for newcomers. vim or emacs more capable and more likely to be used by longer term users of Linux.
  • compilers: These include compilers for C++ and Java. We won’t cover them in this chapter - TBC in updates to this book.
  • apt-get, apt-file, apt-cache and aptitude: are all related to managing software on Linux systems. They’re originally part of a Linux operating system called Debian, therefore the tools all work with ‘Debian’ packages (that contain the various software and files that comprise the package).
  • sudo: enables us to run commands that need what’s known as super-user privileges, where we’re administering the computer and the software that’s installed.

Useful commands

When working with Linux systems for RACHEL much of our time will be spent either working with files, for instance downloading and unpacking RACHEL images, or configuring RACHEL running on Linux where we’re deailing with network and web server configs. We may also want and need to connect to remote computers over a network. In addition, we’ll need to install new software and keep our system up-to-date. For the more intrepid we’ll be working with development tools such as git, building code, etc.

All my examples use the terminal (command-line) interface. Sometimes there are GUI options available, however as we’re likely to need to be able to work using command-lines, particularly when working remotely, we might as well learn the essentials of using commands now.

General commands and tips

First things first, Linux command lines generally support tab-completion, where typing the first part of a command and pressing tab will either complete the command or show possible matches if several exist. Sometimes we need to press two tabs to get a response.

man followed by the name of a Linux command will enable you to read the documentation about that command. For example man ls will enable you to read about the ls (list) command, including the many options.

zip and tar both work with compressed files. Use man to learn more about how to use them :)

Keeping our system up-to-date

We will rely on the various ‘apt’ commands, particularly apt-get.

sudo apt-get update && sudo apt-get upgrade

The double ampersands (&&) tell Linux to only do the command on the right if the command on the left completes successfully.

For more info, here’s the official article on updating and upgrading Raspbian https://www.raspberrypi.org/documentation/raspbian/updating.md

Installing software

There are a trio of useful commands, one you probably need to install so we’ll cover it last. 1 apt-get install is used to install named software packages. You need to get the name exactly correct, including the case. For example sudo apt-get install vim. Vim or VIM wouldn’t be recognised and vim would not be installed. The command needs sudo as it is a powerful command and Linux wants you to confirm you intended to run this command. Sometimes you’ll be asked to confirm the installation with a Y/N prompt. Answer y (pressing enter, or return, also works). Tip, you can include -y in the command line to confirm you’d like Linux to install the software. 2 apt-cache search is used to find packages that match a search term. This can help us to find suitable packages to install e.g. there are several choices of Java, apt-cache search java will list all that match, including many that you may not be interested in. The search runs against a set of sources configured on the current Linux computer. Sometimes we will want to install software from elsewhere, for instance Oracle’s JDK. For now, please search online for information on how to do so (you’ll need to be online anyway to be able to download and install the software). 3 apt-file search apt-file often needs to be installed, so try using sudo apt-get install apt-file -y if the command isn’t found. Once it’s installed, run apt-file update so it can update its currently empty search database. apt-file is able to find which packages contain specific files which is very very helpful when working with building kiwix and other software.

Working with files, partitions and drives

We can explore the drives, partitions and files using various non-destructive commands. These include: * df to show the usage, free space and partitions. * ls to list files in a folder. * tree to show all files and the structure in this folder and all the sub folders. * file provides lots of information about a particular file.

To list the partitions, usage and format use: df -Th

Here’s an example from my laptop

 1 julian@julian-ThinkPad-X220:~/Downloads/rachel$ df -Th
 2 Filesystem     Type      Size  Used Avail Use% Mounted on
 3 /dev/sda5      ext4      194G   92G   93G  50% /
 4 none           tmpfs     4.0K     0  4.0K   0% /sys/fs/cgroup
 5 udev           devtmpfs  3.9G  4.0K  3.9G   1% /dev
 6 tmpfs          tmpfs     787M  1.3M  786M   1% /run
 7 none           tmpfs     5.0M     0  5.0M   0% /run/lock
 8 none           tmpfs     3.9G  604K  3.9G   1% /run/shm
 9 none           tmpfs     100M   80K  100M   1% /run/user
10 /dev/sdc1      vfat      932G   12G  920G   2% /media/julian/Transcend

We can list files with the ls command. Here’s an example for my RACHEL image files:

1 ls -l
2 total 82094540
3 -rw-rw-r-- 1 julian julian 62864228352 May 18 22:59 rachelpi_64EN_3.2.1.img
4 -rw-rw-r-- 1 julian julian 21200564247 May 19 04:12 rachelusb_32EN_3.1.5.zip

The -l option asks ls to list in ‘long’ format, providing more details including the file permissions (who can do what with the files) and the size. These files total about 80GB.

There are several similar commands in Linux, similar in concept to ls which provide information about various aspects of the computer. These include:

  • lsusb: to list the devices connected to USB ports.
  • lsmod: to list various software modules loaded into memory.

Again, use man followed by the command to learn more about what it does and the various command line options they support.

Working with files

The most common command is simply cp to copy files from one place to another. We need to specify both the source files and the destination. * means all the files.

Here is an example where I also used the time command to measure how long the copy took of two very large RACHEL image files (shown above in the ls -l command): 

1 time cp * /media/julian/RACHEL-BACKUP/rachel/
2 
3 real	36m58.934s
4 user	0m1.023s
5 sys	1m30.292s

Formatting filesystems

Here is a useful article that enabled me to reformat a FAT32 partition as exFAT so I could use it to store the large RACHEL image files Formatting USB as exFAT in Ubuntu 13.10

Connecting Computers

Computers only need a network if they want to connect to other computers, including the Internet and computers in a local area network (LAN). There are various aspects to the connection, including how the signals go from one computer to another, and how computers can find each other once they are part of a working network. We will cover the various aspects in this chapter so that we can connect computers together in a working network.

This chapter covers the applications we’re likely to use, and an overview of the core concepts of TCP/IP networking. We’ll start with the commands as these are succinct.

Commands to connect to computers and Android devices

Connecting to computers

The ubiquitous command (and protocol) to connect to another computer over a TCP/IP connection is ssh. ssh is available for each of the operating systems and platforms we use.

  • Linux: ssh is preinstalled.
  • OSX (Mac computers): ssh is preinstalled
  • Windows: Either use putty or install cygwin and then configure it to include ssh` if it’s not included in your setup.
  • Android: ConnectBot is a reasonable Android app that provides basic ssh capabilities. There may be better alternatives.

ssh requires details of the device we want to connect to. We can also provide the username we want to use when we’re connected. Here’s an example: ssh pi@1.1.1.1

We will be asked for the password of that user account. For the current RACHEL images this is rachel. For Raspbian images it’s set to raspberry so try that if you can’t connect using rachel as the password.

Connecting to Android devices

The core command is adb provided in the Android SDK. For Ubuntu this is also available as a Debian package. sudo apt-get install android-tools-adb to install this package.

adb shell connects us to an Android device that has debugging enabled and that’s currently connected using a USB cable. It’s possible to connect using Wi-Fi with some extra configuration.

Commands to transfer files

It’s useful and sometimes essential to be able to transfer files between computers over a network. There are various well-proven commands available. These include scp and rsync on Linux computers (also available for Windows in cygwin) and adb push & adb pull for Android devices connected to a computer.

rsync is particularly useful as it can restart partial downloads. It supports compression while transferring files to reduce the network traffic, which may also reduce costs when the price is based on the bytes transferred.

wget and curl are also worth considering, and I’ve found curl works very well when downloading RACHEL images over the Internet. Here’s an example of using curl to download one of the images:

curl http://rachelfriends.org/rachelpi_64EN/rachelpi2_64EN_3.4.img --retry 999 --retry-max-time 0 -O -C -

Signals and Cables

There are two established ways to connect computers. These are using radio waves and physical cables. Physical networks are almost always using technology called Ethernet. Radio waves vary in frequency, tere are several ‘bands’ of frequencies used for Wi-Fi networks, and others for mobile phone networks such as 3G, CDMA, and 4G (also known as Wi-Max). We seldom need to be concerned with the details unless there are conflicts between several Wi-Fi networks.

Ethernet uses physical cables which connect devices to either a network switch or a network hub. We will cover these in the next section.

Wi-fi networks use one of the 802.11 standards. Again, we seldom need to worry about the details. We mention these standards in case you ever need to troubleshoot connection problems.

Wi-Fis, switches and hubs

One thig is common to all networks, they need some at least one physical device to make the connection to other devices, even when the connection is over Wi-Fi or a mobile network.

  • Wi-Fi: There are two variations of Wi-Fi network. One connects between two computers and may be called an ad-hoc or point-to-point network. This can be useful to share information and materials between those two devices, however it’s a very restricted network as other computers can’t participate so we seldom use them. More common and often more relevant for projects such as RACHEL is an Infrastructure network where one device is the master and controls the network. The RACHEL server is generally configured to be the device that controls and provides a Wi-Fi network.

In some locations there may already be a device configured to provide the Wi-Fi network, if so, then we recommend you configure the RACHEL device to be part of that network rather than trying to provide another network. We also recommend you allocate a fixed IP address for the RACHEL server so that other computers, and the users, can find that server easily and consistently. Allocate an IP address that isn’t part of the pool of loan addresses (provided by the DHCP service). This will reduce the chances of conflicts within the local network.

Finding each other

Each computer and device on a network will need a known IP address in order to communicate with each other and with computers beyond the local network. Sometimes we may end up using an IP address directly, for instance in the web browser of an Android tablet to connect to the RACHEL server.

IP addresses

There are two main standards for IP network addresses, the older ubiquitous IPv4 addresses, such as 192.168.1.102 and the newer IPv6 addresses. We will only cover the IPv4 addresses here as they’re the primary format used by the various computers and other devices we use.

This book aims to explain the essential aspects simply. This means we decided to exclude some precsion and some accuracy of the entire topic. There are plenty of materials available elsewhere that should provide more accurate extra information. Cisco’s CCNA is a well respected qualification for people who want to understand networks well.

IP addresses need to be unique for each device on a network. Otherwise there may be problems that stop one or more of the computers from communcating as they will be competing for what is intended to be a unque address. Computers aren’t very smart about trying to resolve the problems, an administrator of the computer network may need to make changes to the network configuration and/or to the network settins of devices trying to use that network.

IP addresses can be provided on loan, preallocated, or configured on the local computer or device. Typically IP address are configured on servers, such as the RACHEL server, and on network devices such as the Wi-Fi router.

Most other devices request an IP address when they start using a network. They also need to ask if they want the loan to be extended, as each loan has a finite period. Loans vary from several minutes for very crowded networks such as an Internet Cafe, to many months where the administrator doesn’t expect much to change in terms of the computers using that network.

If our assumptions when configuring the loan period are incorrect some devices may not be able to connect to the network.

The addresses are loaned by a service called DHCP. In some cases, such as when the RACHEL server provides the Wi-Fi network, these addresses need to be configured on that server (the preconfigured RACHEL servers are already configured to do so). Wi-Fi routers generally have a web-based administrator’s interface where the set of addresses are configured.

Common IP addresses for RACHEL Servers

There are at least 3 IP addresses which the project has used in RACHEL Pi builds. These include:

  • 1.1.1.1
  • 10.10.10.10
  • 192.168.1.10

If you’re computer or device is connected to the Wi-Fi provided by RACHEL Pi you can use your IP address to determine (or guess) the IP address of the RACHEL Server. You may be able to check your IP address in the GUI, or on Windows, OSX and Linux, from the command line using either ipconfig /all (on Windows) or ifconfig on other operating systems.

Local and foreign networks

Conceptually there are local addresses and foreign ones, somewhere else on another network. Generally we want our computers to share a common, single logical local network. This means that most of the IP address will be the same, this is always the first part of the address, reading from the left. For instance they may all begin with 192.168.1 where only the final part is unique.

Getting a RACHEL Server running on an existing network

If there is an existing computer network available we may want to configure our devices to share that network. To do so we would need to know various parameters such as the name of the Wi-Fi network, any password, etc. For devices where we want to assign a fixed IP address we will also need to know what IP address to use, and other details such as the subnet, the default gateway and the DNS server. We will cover each of these later in this chapter.

Raspberry Pi Model A’s

Raspberry Pi model A’s don’t have an Ethernet port, so they’ll need to connect to the network using a USB Network adapter. Typically this is a Wi-Fi adapter such as the WiPi. The current RACHEL images configure the WiFi adapter as a network HotSpot, where it assigns IP addresses. Typically the existing network will also have a machine (typically the router to the Internet) that assigns IP addresses. Generally you’ll want to reconfigure the RACHEL so that it’s no longer a HotSpot, instead it would use an IP address from the existing Network.

Raspberry Pi Model B’s

Raspberry Pi Model B’s have an Ethernet port. Sometimes we may add an additional USB Wi-Fi adapter. Again the current RACHEL images configure the WiFi adapter as a network HotSpot. This means the Pi (and therefore RACHEL) can potentially be used on either network provided we also configure the Ethernet port so it has an IP address of the same network as the existing (non RACHEL) network.

Obtaining an IP address

The network port could use a static IP address - one that you configure the Raspberry Pi to use, or it could be configured to request and receive an IP address chosen by a service called DHCP. This service will be running on the machine that assigns IP addresses.

Finding a Raspberry Pi on the local network

If you assigned a static IP address to a Raspberry Pi, you know enough to reach it without further information. for instance you could SSH in to the operating system running on the Pi (I’m assuming SSH has been enabled and configred, as it is on Rasberian and RACHEL images), and you could connect to RACHEL with a web browser using by adding the prefix http:// to the IP address of the Pi.

If you don’t know the IP address of the Pi, there are various ways to find it. These include:

  • Using the Web interface of the router, which will often list which DHCP IP addresses are being used.
  • A network scan using a command-line utility (nmap, or ping), etc.
  • Connect a keyboard and monitor to the Pi, login (probably using the pi account) and run ifconfig from the command line, then search for the IP address in the results. (there are ways to refine this command e.g. piping the output as follows ifconfig | grep inet and/or specifying the interface e.g. eth0
  • Try using ascending IP addresses from the range that’s been assigned, while this can be very time-consuming and tedious, it generally works eventually and can enable you to find the RACHEL software when no other tools or techniques are available

Each network adapter has a low-level address, called a MAC address. The first half identifies the manufacturer of the network adapter. With practice we can learn to recognise the prefix for network adapters used by the Pi. TBC add commonly used prefixes for the Ethernet and WiPi adapters.

Subnets

The subnet is a filter that matches the common part of the addresses, at least for the purposes of this book and our projects. The popular term is the subnet mask, which can be represented in several forms. These include the .dot notation e.g. 192.168.1.x and /24 which represents the number of bits in the network address that remain constant for each address in this network.

Default Gateways

Computers address each other directly when they share a network. To reach computers in other networks they forward all their requests to a network address known as the ‘default gateway’. This may be a device such as a router, or another computer such as a Raspberry Pi. That device is responsible for trying to find a route to connect the computer making the request to the relevant remote device, often another computer such as a web server.

By convention, default gateways often have an IP address that either ends in .1 or .254 Computers need to be told what the address of the defult gateway is. They get this information with their IP address.

Names, URLs and IP addresses

Humans generally prefer to use names than numbers to address things. However, computers including their web browsers actually use the IP address as a number. They need to obtain the correct IP address for whatever name is being used. There are several ways to do so, the most common are either using a DNS Server or a small text file locally on that device. When we use a mobile network or an existing Wi-Fi network they often ‘know’ where to find the DNS Server which then performs the mapping between the name and the IP address that matches that name.

Sometimes we may want to configure some of our computers to override several of the names being used so that informaton can be provided within the local network rather than needing an Internet connection. These local settings can help provide updates of software and materials. They can also help reduce undesireable use of potentially expensive and scarce Internet connections, for instance to restrict access to sites such as social networks (Facebook, Twitter, Instagram, etc).

Configuring a RACHEL Server

  • TBC: until we add the relevant material to this book, a good place to obtain the information is on the wiki of the ka-lite project https://github.com/learningequality/ka-lite

Advanced topics

Sometimes we need to do some more complicated networking in order to get our work done. One example is connecting a Raspberry Pi (running as a Wi-Fi hotspot) to the Internet through a laptop connected over Wi-Fi. I have had to use this approach to update RACHEL Pi, for instance to update the kernel files so the image will run on a new model of Raspberry Pi.

Reverse-proxy on a computer

When a computer is configured as a reverse-proxy, it allows connected devices to use it to connect to another network. The computer also needs to be configured as a DHCP server (to provide the Pi with an IP address to its Ethernet port, the Wi-Fi adapter on the Pi is being used as a Wi-Fi hotspot).

The configuration of a reverse-proxy is specific to each desktop operating system. TBC add some links here.

Here’s an overview of how to configure a reverse-proxy on OSX http://makezine.com/2012/07/16/use-your-mac-laptop-as-a-wireless-proxy-for-raspberry-pi/ ##Further reading I found a very helpful guide on IP protocols and how all this stuff works, at http://ipprimer.com/#/ perhaps you’ll find it useful too?

Troubleshooting

Software can help us troubleshoot configuration problems, and runtime errors. We can also investigate logs to understand more of how the software is behaving.

Each operating system has particular tools, here’s a brief table for comparison.

Utility Android Linux Windows
System logs adb logcat dmesg eventvwr
System health ? top Task Manager
IP info ? ifconfig ipconfig

Where practical have spares of each item available. This enables us to test whether the fault is related to a specific item, or not. Be careful as failures may be a symptom of another problem - the cause. And sometimes, unfortunately, the cause may cause multiple and repeated faitures of other items. For instance, a short in a power lead may cause previously working fuses to fail.

Software utilities such as grep can help us filter and search files, including logs, for indicators of [possible] problems. However,grep needs to be downloaded and installed on Windows computers, or an existing utility called findstr can be used.

USB RACHEL

RACHEL can be used from a USB drive. Much of the contents can be used directly, for instance, by accessing the files directly in a Web Browser (using the file:// prefix), or ZIM files can be read and used in the Kiwix application.. However, some of the software, for instance Khan Academy Lite needs to run as a web server. TBC check what won’t work if the Web Server and MySQL functionality aren’t used.

USB RACHEL includes an embedded web server and database that can be used on Windows Computers. We will describe these in the Running on Windows chapter.

There are lots of files, which take a lot of space. The choice of file format on the computer can affect performance and whether the files can all fit on a single drive. We recommend exFAT (check) provided the computer is able to read and use this format. The minimum requirement for Microsoft Windows is XP with Service Pack 2 or 3 and the following update from Microsoft https://support.microsoft.com/en-us/kb/955704

RACHEL on Android

RACHEL for Android is currently an experimental project. The software works on many devices and versions of Android. It essentially consists of a small web server and a small database server (as does RACHEL for Windows computers) together with the many files comprising RACHEL USB release. The Android device can be configured to share RACHEL with other people on the same Wi-Fi network.

The device needs enough free storage available to hold these files (around 22GB) which is more than most devices have. Thankfully many devices accept a micro SD-Card, so the files can be copied onto a 32GB and used in the micro SD-Card slot. The quality and performance of the micro SD-Card may affect how well RACHEL behaves for the user(s).

Downloading, installing, and adding contents are all currently documented online Installing RACHEL on an Andoid Tablet, etc

On Android RACHEL relies on a small web server and has a mySQL database running in the background. The embedded web server means the app is unlikely to be approved by Google’s App Store. The contents need to be available on the device. They can be copied to internal storage (known as /sdcard/ for historical Android reasons), or on removeable storage such as a micro-SD card or USB storage connected using OTG (On The Go) protocol.

Notes

  • TBC migrate the instructions here?
  • Check also the comment about restrictions in some versions of Android (4.4?) that cause RACHEL problems on these versions of Android.

RACHEL on Linux (Ubuntu)

We currently package RACHEL for Windows and Raspberry Pi’s. We also provide an Android app with some instructions on how to get RACHEL working on Android. This chapter is intended to help you get RACHEL working on Linux computers that use Debian packages, and more specifically on Ubuntu. The material is being developed using Ubuntu Desktop 10.04 LTS, you may need to tweak these instructions for other versions of Ubuntu and for other flavours of Debian-based Linux environments.

Installing the web server

Caveat: I had already installed nginx earlier, so skipped some of the steps in the following article. Please read the following guide How To Install Linux, nginx, MySQL, PHP (LEMP) stack on Ubuntu 14.04 and follow the instructions. I made few changes for RACHEL, these include:

  • changing the filename from default to rachel
  • leaving the servername set to localhost
  • leaving cgi.fix_pathinfo commented out (at least until I complete these instructions)

Example configuration files

Here are some examples from my current Ubuntu laptop.

/etc/nginx/sites-available/rachel

 1 server {
 2         listen 80 default_server;
 3         listen [::]:80 default_server ipv6only=on;
 4 
 5         root  /home/julian/Downloads/rachel/rachelusb_32EN_3.1.5/rachelusb_32EN_\
 6 3.1.4/RACHEL/bin/www;
 7 
 8         index index.php index.html index.htm;
 9 
10         # Make site accessible from http://localhost/
11         server_name localhost;
12 
13         location / {
14                 # First attempt to serve request as file, then
15                 # as directory, then fall back to displaying a 404.
16                 try_files $uri $uri/ =404;
17         }
18 
19         error_page 404 /404.html;
20 
21         # redirect server error pages to the static page /50x.html
22         error_page 500 502 503 504 /50x.html;
23         location = /50x.html {
24                 root /usr/share/nginx/html;
25         }
26 
27         # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
28         #
29         location ~ \.php$ {
30                 fastcgi_split_path_info ^(.+\.php)(/.+)$;
31                 try_files $uri =404;
32                 fastcgi_pass unix:/var/run/php5-fpm.sock;
33                 fastcgi_index index.php;
34                 fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
35                 include fastcgi_params;
36         }
37 }

After some fiddling (I’ll find out when I repeat these instructions on a clean machine) I managed to launch and run RACHEL in a web browser locally on the same machine. We would need to amend the security settings in nginx to allow remote access, TBC determine these changes. Limitations:

  • The full-text search from the RACHEL homepage doesn’t work at this stage, otherwise the web contents are useable, including the videos. We will get the full text search working in the next section.

Enabling full-text search

Full-text search in RACHEL enables users to search and find contents that match their search term(s). RACHEL currently uses rsphider which in turn relies on mySQL. At the time of writing, RACHEL for Windows uses mySQL 5.6 while Ubuntu 14.04 LTS provides 5.5 by default. I discovered I needed to explicitly install 5.6 to get mySQL to accept the RACHEL mySQL databases.

Installing mySQL 5.6

sudo apt-get install mysql-server-5.6

We recommend running the script to enhance security:

Here was the error I got when I used 5.5 when starting mysqld : status: Env must be KEY=VALUE pairs ###Migrating RACHEL rsphider mySQL database files NB: This process doesn’t follow the recommended mySQL practices, we should be able to find more robust and standardised methods in future. For now, this seems to work and we probably should find ways to script the creating, maintenance and updating of the full-text database as a more strategic and elegant approach.

Overview of steps:

  • Hide the default databases: in /var/lib run sudo mv mysql mysql.local
  • Make a fresh empty location for our database files: sudo mkdir /var/lib/mysql
  • Copy RACHEL’s mySQL database files across: sudo cp -r /home/julian/Downloads/rachel/rachelusb_32EN_3.1.5/rachelusb_32EN_3.1.4/RACHEL/bin/bin/database/mysql-5.6.20/data/* /var/lib/mysql/
  • Run various sudo chown commands so the paths in `/var/lib/mysql/’ See below for examples.
  • Start mySQL.
  • Try the search in your web browser.
1 sudo chown -R mysql:mysql ib_logfile0
2 sudo chown -R mysql:mysql ib_logfile1
3 sudo chown -R mysql:mysql ibdata1 
4 sudo chown -R mysql:mysql sphider_plus 
5 sudo chown -R mysql:mysql performance_schema
6 sudo chown -R mysql:mysql .
7 sudo chown -R mysql:root mysql

The following reflects the current permissions (some were applied automatically after upgrading from mySQL 5.5 to 5.6)

 1 sudo ls -la /var/lib/mysql
 2 drwx------  6 mysql mysql     4096 May 24 20:51 .
 3 drwxr-xr-x 78 root  root      4096 May 24 20:50 ..
 4 -rw-r--r--  1 mysql mysql       56 May 24 20:42 auto.cnf
 5 -rw-r--r--  1 root  root         0 May 24 20:50 debian-5.6.flag
 6 -rw-r--r--  1 mysql mysql 12582912 May 24 20:50 ibdata1
 7 -rw-r--r--  1 mysql mysql 50331648 May 24 20:51 ib_logfile0
 8 -rw-r--r--  1 mysql mysql 50331648 May 24 20:42 ib_logfile1
 9 drwx------  2 mysql mysql     4096 May 24 20:51 mysql
10 -rw-rw----  1 root  root         6 May 24 20:51 mysql_upgrade_info
11 -rw-r--r--  1 mysql mysql     6425 May 24 20:42 nuc.err
12 -rw-r--r--  1 mysql mysql        5 May 24 20:42 nuc.pid
13 drwx------  2 mysql mysql     4096 May 24 20:51 performance_schema
14 drwxr-xr-x  2 mysql mysql     4096 May 24 20:42 sphider_plus
15 drwxr-xr-x  2 mysql mysql     4096 May 24 20:42 test

Sample Commands

http://localhost/rsphider/search.php?query_t=ethiopia&submit=Search&search=1&db=0&prefix=0

sudo service mysql start or use mysqld -v which provides more detailed information, especially useful if mySQL fails to start. dmesg | grep mysql is also useful to find reported problems when mySQL fails to start. sudo service mysql status mysql --user root -p then enter your password interactively, so it’s not stored in the command history mysql> show databases;

Extra notes

The following web site provides some useful ideas, including the test.php page. However the security settings differ from those used above and I prefer not to use them as they seem too permissive (too open). http://wiki.nginx.org/PHPFcgiExample

Known Unknowns

There are various things I know I don’t know. They’re not necessarily limited to running RACHEL on Linux, however I realised them during the process and they might as well be mentioned here.

  • How to install and configure RACHEL cleanly on Ubuntu (the above is distilled from the things I did which included some backtracking)
  • How the full-text index is created and updated
  • What the limitations are of the full-text search
  • What the appropriate security settings should be
  • How to configure nginx for access from other machines (where it can cope with the IP address changing and not having or using DNS)
  • What to do with the zipped sphider_plus.zip file used on Android

RACHEL on Windows

USB RACHEL includes an embedded web server and database that can be used on Windows Computers.

There are a couple of ways RACHEL can be used. The two most common ones are:

  • Running directly from the USB drive. This is fast, and an excellent way to test whether RACHEL will work on the particular Windows computer
  • Running from the computer’s drive. Initially this takes much longer to setup, particularly on older computers that predate USB 3.0 drives, etc where the files may take many hours to copy from the USB drive. However, once the files are safely copied onto the computer RACHEL can be used on that computer without needing anything else and the USB drive can then be used with/on other computers.

Programs used by RACHEL on Windows

  • UwAmp provides the main web server
  • MySQL is the database server, used to support the full-text search feature in RACHEL
  • tasklist.exe (called by Start RACHEL.bat) which tries to detect whether the web server is already running. NB: This program is not available on Windows XP Home Edition, so we need to find a solution.

If you are running Windows Security software you may be asked whether you want to unblock Apache HTTP Server and then mysqld - if so, select unblock.

TBC Add section(s) on what’s involved in terms of running the software: web server, MySQL, etc.

When RACHEL fails to start

RACHEL is generally reliable on Windows computers. However there may be some configuration issues to resolve in order to get RACHEL running, or there may be problems that need fixing.

Troubleshooting tips

First things first - please pay attention and note down the errors and warnings. That way you’ll be able to ask for help and provide these details to whoever you ask. They’re more likely to be able to help you diagnose the problems to help you get the software working again.

Problems copying files (and how to check)

RACHEL USB currently consists of roughly 250,000 files. Sometimes the copy can fail to copy all the files, or some files may be advertently moved or deleted.

  • Right-click on folders in Windows Explorer and check the Properties of the folder. Allow Windows time to calculate the results which can take several minutes for the entire RACHEL package or any folder that includes the main contents at bin\www e.g. RACHEL or RACHEL\bin both contain the www folder in their results.
  • Run dir /s command in a command window

Compare and contrast the number of files and folders between several computers with RACHEL installed, or compare with the files on the source USB drive. There may be some differences in the overall sizes depending on the filesystem used.

xcopy can be run from the command line to selectively copy missing files. The key parameters are:

  • /s to include sub-folders (also known as sub-directories)
  • /d to tell xcopy to only copy files that are newer (which also copies missing files across since they’re treated as newer).

Sample command line. Run from the equivalent location on the local computer e.g. C:\Users\julian\: xcopy E:\*.* . /s/d This assumes we’ve already created a rachel… folder locally in my default folder. There are too many differences in Windows versions for me to easily document all the variations (sorry).

Configuring antivirus software

Some antivirus software blocks the programs used by RACHEL for Windows as a security precaution. You need to configure the antivirus software so that it doesn’t block these programs. How to configure the various programs differs for each program. Some have a concept of exceptions, or exclusions, where you can tell the antivirus software to effectively ignore or trust programs in a particular folder.

RACHEL\bin is where the various programs are located that RACHEL needs to run. Within that folder, and nested several folders deep are the MySQL program files which also need to run. They are in RACHEL\bin\bin\database\mysql-5.6.20\bin (at least in the release of RACHEL when this section was written). The folder name may change in future if and when we update the version of MySQL that’s used by RACHEL.

Avast antivirus

How to configure permanent exclusions http://classroom.synonym.com/make-exclusions-avast-16518.html

TBC Add additional antivirus programs

Windows Task Manager

Microsoft provides a powerful small program called Task Manager that provides information about the running programs, processes, and what’s being used currently (including memory and network). It can also be used to kill (stop) running programs. We may need to use this to kill the web server or MySQL if they’ve not started properly (they’re designed to keep running).

Task Manager can be started in various ways. Here are several of these ways:

  • Press the combination of these 3 keys together at the same time: Ctrl+Alt+Esc
  • Right-click on the Windows Bar on the desktop screen (where the Start button is on the bottom left, typically) and select Start Task Manager
  • Press the combination of Ctrl+Alt+Del (to lock, or reboot the computer) and select Task Manager from the dialog box that’s displayed

Security Settings and Web Browsers

  • Internet Explorer’s Security Zones. Intranet security for RACHEL on localhost.

Choice of Web Browser

The ultimate choice depends on various factors, however, particularly for Windows XP then either Google Chrome or Firefox are good options if they are available to install.

http://lifehacker.com/download-full-firefox-and-chrome-installers-for-offline-1570335126 explains how to download full versions of Chrome and Firefox so we can install them offline.

User and Administrator roles

  • Different roles help improve the security and reliability of computers by limiting who can run which programs when using User accounts. The Administrator role has full power and access to everything, however this means that anyone running as an Administrator could easily make a mess of the computer, and it’s easier for viruses to worm themselves into the operating system.
  • User accounts seldom have access to change security settings of antivirus software, etc. So some of these changes need to be done using an Administrator’s account.

The default Administrator account is called ‘Administrator’. This is not listed by default in the login options. Pressing Ctrl+Alt+Del key combination twice at the login prompt can let you login using this account provided you know the password, of course.

Resetting the Administrator’s password (or other passwords).

There are various ways to reset passwords. Here are suggestions. For now, please search for details online.

  • Boot CD-ROMs with a password reset utility
  • Administrators can set the password for user accounts
  • Accounts can update their own password

Optimising performance

Many of the computers where we want to run RACHEL are relatively old, overworked and underpowered compared to what we might like. The performance of RACHEL may be particularly poor (slow) when the computer doesn’t have enough RAM memory for the set of running programs. For instance, on a small netbook from 2009 the computer has 1GB of RAM however the RAM needed is over 1.5GB to run RACHEL and Google Chrome (without any other applications).

Windows includes utilities to tune the performance. Microsoft provides various additional utilities to download, including: PowerToys can help tweak the UI, and SysInternals utilities enable much greater control and understanding of what your operating system is doing. Microsoft provide training online for SysInternals at https://technet.microsoft.com/en-gb/sysinternals/bb469930 - well worth watching when it’s practical to do so.

Maintaining RACHEL

Computers need to be maintained in good working order so they can continue to operate efficiently and effectively. We also need to make precautionary backups, particularly of databases, and of other data generated such as usage logs if we want to be able to understand how the software is being used, etc. Finally, we need to protect the integrity of the computers by shutting them down cleanly before power is removed. Android Tablets and other devices with integrated batteries may be configured to do this by their operating system. For other devices, including Raspberry Pi’s we may need to setup and add appropriate configuration scripts, etc. to suit it’s power source(s).

Maintaining RACHEL on Raspberry Pis

this is a rough set of notes, probably worth expanding. Here are some things to consider to help operate the software longer term:

  1. Backing up the SQLite database on a regular basis (protect the usage records, accounts, etc.)
  2. Expanding the partition to use all the available space on the 128GB memory card (we’re currently only using about 64GB)
  3. Providing ways to safely shutdown the ka-lite software (otherwise there is a risk that the memory card will be corrupted and the software fail to work after power outages for the raspberry pi).
  4. Updating the ka-lite software (using git commands)
  5. Adding local contents and materials (possible through the Rachel Web User Interface AFAIK).
  6. Installing kiwix to replace the current wikipedia contents (I will need to send you instructions which I’ve yet to write).

External Information

Scaling and Performance

Installing RACHEL once is enough for many people, and can take many hours from start to finish. However, when we want to install RACHEL on more computers, the challenge of scaling and working efficiently become more important.

Variations in performance

Many factors can affect performance. Ultimately we want to work efficiently and effectively and for users to be able to use the system productively without material delays or glitches. Factors include:

  • Usage patterns
  • Hardware used
  • Software drivers and configuration settings

Of USB Drives

The performance of file copies vary massively. The model of USB storage can affect the performance several fold, as can the type of USB interface on the computer (and its performance). As a guideline USB 3.0 drives when used with USB 3.0 USB port will give the best throughput.

My current computer, an X220 i5 only has 3 USB 2.0 ports. All the USB drives support USB 3.0.

time unzip rachelusb_32EN_3.1.5.zip -d /media/julian/USB\ DISK/

Results for 2 runs, each with brand new USB drives.

measurement Integral 32GB drive Sandisk Kingston 64GB drive
real 70m59.300s 49m28.714s 99m2.430s
user 13m46.886s 14m2.900s 14m15.597s
sys 8m21.883s 8m58.815s 9m21.392s
real 65m30.835s 47m3.925s 88m45.254s
user 4m54.700s 7m58.874s 13m40.720s
sys 3m11.843s 4m30.248s 7m56.550s

Even in the first test comparison there’s a 2:1 ratio between the drives, in the same USB slot. Now imagine trying to prepare 10, 50 or even 1,000 USB drives to ship to projects!

Of Raspberry Pi models

  • B+ Model 2: is the current performance winner in terms of capacity and throughput.
  • B in terms of storage capacity as SD-Cards are available with approximately 2x the capacity than micro SD-Cards currently. (This situation will probably reverse within 12 months)
  • A+ in terms of power usage and overall dimensions.

Testing and measuring performance

A big, rich topic. TBC.

  • HTTP and Browser-based automated testing
  • Performance measurement utilities e.g. top
  • time

Remember to make notes (there are several examples in this chapter and elsewhere in the book where we use time and learned more about the performance of aspects of the system.

Performance Analysis

Another TBC topic. However we can analyse log files to infer performance data amongst other techniques.

Working with Raspberry Pis

When using RACHEL on Raspberry Pi’s there are several things that are useful to know. These include how to see what’s happening and how to provide inputs to the Pi. Also, we may need to update the kernel of the operating system in order to use an existing image on newer models of Raspberry Pi (and there’ve been 3 in the last 12 months) otherwise your current RACHEL image will not boot in the newer devices.

Interacting with a Pi

We can connect a Raspberry Pi directly, using a USB keyboard, and optionally a USB mouse. For the display we can connect directly to an HDMI display (which includes many recent flatscreen televisions), composite video (supported on older generation TVs and some small flat screens intended for use in cars, or using a special HDMI to VGA cable. We recommend using HDMI where practical as the quality is far superiour and you’re unlikely to need to mess about with finding suitable cables or adapters the other options need.

Another way to connect is over a network. Model B Pi’s include an Ethernet port, and both Model A’s and Model B’s support USB Wi-Fi adapters. The operating system (Raspbian) is preconfigured to support the WiPi adapter which is one of the reasons we recommend and use this adapter.

  • Raspbian images: have ssh enabled by default. They will request an IP address using DHCP so can connect to an existing logical network. You will need to find out which IP address the Pi has been assigned, this will be covered in the Connecting Computers chapter.
  • RACHEL images: also have ssh enabled. They are configured as a DHCP server, so if we connect to a Pi running standard RACHEL builds we will be allocated an IP address for us. Using network commands we can determine the IP address of the Pi and connect using ssh.

Updating the kernel files

TBC

Working with materials

The power of RACHEL and related projects comes from the materials made available for people to use.

Some practical context (a personal perspective)

In the last few years and during my travels I’ve come to the conclusion that the packaging and distribution of materials is one of the biggest challenges for RACHEL and similar projects that aim to provide contents to the vast majority of people and locations that don’t have reliable, high speed Internet access.

Here are my heuristics I’m working towards:

  1. the maximum chunk should be designed to fit on a USB stick one could buy at any tourist resort or medium sized city globally e.g. climbing the Andes, at a beach resort in India, in a shopping mall in Nairobi, etc. That’s somewhere either 8GB or 16GB currently based on my admittedly very limited research (I check wherever I go in the world). I also consider ways to reduce the chunk size to one that’d fit (and be useful[1]) on a DVD-ROM and even a CD-ROM. The aim is to be able to deliver whatever we want on one or more practical, available pieces of hardware (and by that I include USB sticks, etc).
  2. Modularity helps reduce the update size dramatically. Until recently folk had to redownload the entire image if they want the latest updates. <dev.worldpossible.org/mods> now allows people to download individual modules (using rsync, which is available in cygwin for Windows users). I’d like to get even more modular.
  3. Compression also helps. Emmanuel of Kiwix is helping me with our PhET experiment to see if we can script the packaging of PhET. The principles and the commands we end up using may be generally applicable for rich HTML5 apps.

Money to pay good developers / devops / system administrator people may help us make better progress. Also, the industry is solving these problems for commercial apps, content delivery, etc. so let’s investigate the practices that seem to work and try to apply them as a starting point.

[1] having smaller usable chunks of content and software can also help people who simply don’t have the luxury of a device with 32GB available e.g. where a useful subset of RACHEL contents are being deployed by a project in Kenya who only had a few GB available; and I have various cheap Android tablets with roughly either 3GB or 6GB available from new (depending on whether they were equipped with 4GB or 8GB storage). Kiwix and zim files are one approach to package and distribute much of the materials.

Finding stuff

Making ‘offline copies’ of web sites

In principle it’s possible to create offline copies of sites. Technically, the key proviso is: provided the contents can work locally (without needing access to that organisation’s servers, databases, etc) then it should be possible. However before we start trying to download someone’s web site, we need to ensure we have their permission.

TBC more on ways to copy/download online materials

  • PhET github example
  • How Kiwix does it
  • Web content crawlers
  • Downloaders e.g. for YouTube videos. Picking the most suitable format for videos, documents etc.

Permission to use existing stuff

Writing new stuff

  • Blogging software, such as WordPress, can be added to computers running RACHEL so that people can create and share their work locally. Potentially they could also make it available on the Internet if they have a working connection.
  • Free software called Pachyderm can be used to create presentations in HTML5 https://sourceforge.net/p/pachyderm/wiki/P3_Whats_New/. We can then package and use the recorded presentations as part of our projects.

Collaborating vs. Single author

Enhancing public materials

Preparing and packaging materials

‘Local contents’ is a good place to share ad-hoc materials, for instance individual PDF files. however it’s not well suited to packaging a collection of content such as a set of videos, lessons, exercises, etc.

Packaging materials enables them to become a distinct entity, where they can be distributed, incorporated, and used - hopefully as a cohesive body of stuff. There are various ways the material can be packaged, there are some inevitable variations needed so they are acceptable to the different programs that will read and present them.

RACHEL Modules

RACHEL consists of modules that contain various materials, such as videos, web contents, etc. Each package has a consistent structure: each is a separate folder, with a file called index.htmlf.

index.htmlf contains HTML, part of a web page, a fragment. The fragment starts with a <div> tag and ends with the closing </div> tag. The div tag has a class of indexmodule which helps keep the styling consistent.

Here’s a very simple example, from the current music theory module:

1 <div class="indexmodule">
2     <a href="<?php echo $dir ?>/index.html"><img src="<?php echo $dir ?>/musicth\
3 eorylogo.jpg" alt="Music Theory"></a>
4     <h2><a href="<?php echo $dir ?>/index.html">Music Theory</a></h2>
5     <p>Comprehensive music theory lessons and exercises to begin your journey in\
6 to the realm of music.</p>
7 </div>

you may notice it references an index.html page. Most (but not all) modules have an index.html page. This index.html makes the contents easier to navigate from a File Explorer program such as Microsoft Explorer or Nautilus on Ubuntu.

The modules are sub-folders of the following folder /RACHEL/bin/www/modules/ (in Windows the / are replaced by ).

Here’s the current set of fragments for the packages provided with USB RACHEL 32GB.

 1 ./powertyping/index.htmlf
 2 ./medline_plus/index.htmlf
 3 ./hesperian_health/index.htmlf
 4 ./math_expression/index.htmlf
 5 ./khan_academy/index.htmlf
 6 ./ebooks/index.htmlf
 7 ./understanding_algebra/index.htmlf
 8 ./khan_health/index.htmlf
 9 ./olpc/index.htmlf
10 ./ck12/index.htmlf
11 ./practical_action/index.htmlf
12 ./infonet/index.htmlf
13 ./iicba/index.htmlf
14 ./musictheory/index.htmlf
15 ./wikipedia_for_schools/index.htmlf
16 ./scratch/index.htmlf

Packaging for Kiwix & kiwix-serve

Kiwix and kiwix-serve use a common file format called zim, or zimfiles. These have an extension of .zim. The materials should be usable in a modern web browser, either served by a web server (we test with nginx), or using the file:// protocol (which is what happens if you open an html file in Windows Explorer, and the equivalent for other operating systems. A program called zimwriterfs is used to package the files.

Using Contents management software

  • We could try a simple contents management program such as Statamic http://statamic.com/learn (Thank you Damien for the suggestion).

Packaging for RACHEL php Server

RACHEL Web Server currently runs a small php script (index.php) that scans sub-folders for files with an extension of .htmlf. If you create one of these files (called index.htmlf) in the root of the package, then your contents can be integrated in the homepage of RACHEL. The file needs to contain an HTML fragment, where everything is wrapped in a <div> tag.

TBC add an example skeleton here.

Preparing materials

While we can simply package the files, preparating the structure, removing unnecessary files, and making the materials more consistent in terms of layout, naming, etc. can help make the materials easier to maintain and manage. Also, consider that some users will have relatively slow, small computers, to read or interact with your materials, so you may want to check the materials are usable on low-cost underpowered tablet devices, etc.

Shrinking the materials

  • Image compression - image resolution that’s still usable and attractive for the target devices.
  • Removing bulky stuff - ask is it necessary, can we device a low-bandwidth alternative, etc.

Intellectual Property

  • Intellectual Property laws
  • Creative Commons Licensing

Publishing

Locally

Online

To upstream projects including RACHEL

Working with software development

There are parallels in working with materials and working with the underlying software. For instance, the materials are public and available to modify, enhance and use. There are explicit and implicit working practices, the etiquette, where aligning your practices to the expectations of the various communities can make a massive difference to your effectiveness and to the progress you and the projects make.

Opensource projects

Virtually all the software we currently use as part of RACHEL is opensource software. The current exceptions include the embedded web server used by RACHEL for Android. Opensource fits well with a collaborative distributed way for volunteers, NGO’s, educational establishments, etc. to work effectively. Some governments, including the UK and USA are deliberately choosing to publish more of their software as opensource projects as it’s been paid for by their taxpayers and citizens.

Opensource means you have greater flexibility to work with the software, particularly in the longer term. The tradeoff is you’re expected (often implicitly) to be willing to do some more work yourself or to sponsor competent people to contribute on your behalf. Learning how to be effective can take longer than common software for Microsoft Windows for various reasons. For RACHEL, ka-lite, kiwix, etc the trade-offs are well worth the effort.

Opensource licenses

In brief, you need to be aware of, and comply with the relevant opensource license used by whatever opensource software you want to use, and potentially modify or contribute to. For the current projects used by RACHEL the licenses are unlikely to cause material restrictions.

See http://opensource.org/ which has a good overview as well as lots of much more indepth materials.

Most opensource projects now use Git as the source control tool of choice. The source code is stored in a Git Respository, and the commands used by developers generally start with ‘git’ together with some additional parameters.

The most popular locations for opensource projects include:

  • github.com - used by ka-lite, this book, and many related projects
  • sourceforge.net - used by kiwix

Some projects have their own location for their source code, and some project have several locations, some long neglected.

Finding information

  • Stack Exchange web sites and particularly http://stackoverflow.com are excellent places to find answers to programming questions. You can also ask questions, participate, help others, etc.
  • Development groups, mailing lists, etc. Each project typically has at least one central location for the developers to correspond. Some also use an old chat service called IRC where they ‘chat’ in real time.
  • FAQ’s (Frequently Asked Questions) are written by the development team to help newcomers find answers to common questions rather than ‘bug’ the mailing lists, etc. They are unlikely to be all you need, however people expect you’ll have at least skimmed through the FAQ’s as a prerequisite to expecting a response to your question. In other words, if you’re asking a question where the project team believe you’d have found the answer in the FAQ don’t expect them to respond.
  • Project Wikis, aimed at developers. Generally these are more complete, or in-depth answers than on an FAQ. ka-lite has a particularly complete wiki for developers.
  • Bug reports and Feature requests: Someone else may have posted a similar problem, question request, or concern; so it’s worth reading the current bug reports and feature requests before filing one. They may also help you deal with your more general question such as whether something is expected to work or not.

Etiquette

People appreciate you demonstrating you’re polite, done your research, etc. and are able to clearly describe your situation & request or concern. Occasionally you may get a grumpy response, especially if you appear to be lazy or asking for someone else to ‘do your work’. Whatever happens, the best approach for you and the situation is to be calm, polite and humble. Sometimes others will respond more positively, especially if they can detect your good manners and attitude.

How you can contribute and benefit

Translations and localisations

Testing, bug reporting

Bug fixing and enhancements

Custom variations

Sometimes you’ll want to develop the software to do different things than the original project team. Provided you’re willing to accept the extra workload and burden of bringing another software project to life, opensource projects will permit you to create a derivative work (project) provided you comply with the software license agreement of the upstream project. One of the longer-term challenges will be managing and merging updates from the upstream project(s).

RACHEL is created, maintained and enhanced by various volunteers. The various projects RACHEL combines also rely mainly on voluneers. Perhaps you’d like to get involved in helping create new material for RACHEL, maintain materials and the software programs, and enhance them?

All the projects welcome help, many have guides so you can get started and learn what to do in order to contribute effectively.

Contributing to KA-Lite

Translating KA-Lite Contributing to the code (software)

Contributing to Kiwix

A good place to learn about known projects for Kiwix is http://www.kiwix.org/wiki/Projects

Contributing to RACHEL

TBC 2015-05-23

This book does not contain everything, it’d be unwise to even try to do so. Instead, here are pointers to other information that may be relevant, interesting, and even helpful for you.

Reaching humans involved in the project

There are various sets of loosely connnected people who are collaborating on various aspects of RACHEL and the upstream projects. When we know how to reach them we publish the details here.

There’s more to add, I’ll do so as and when I (re-)discover them.

Good manners online

Most of us contribute our time freely, in addition to whatever else we do to earn a living and cope with life’s challenges. Like you, we have limited time available to get involved and help people. To help increase the chances of people being able to help you, please do some initial research to see if your questions have been answered online either in one of the groups or in a FAQ (most projects have one), README document, etc. Also please be specific in your request, bug report, etc. and show your workings i.e. explain what you’ve tried and what you expect(ed) to happen.

Installation and development guides

Software Sources

Downloads of complete RACHEL images

Contents

Examples of software online

Paraphenalia

Projects need more than a few Tablet computers. And in particular, those who setup, configure and maintain the technology need and benefit from a wide range of equipment and software.

  • A lightweight, compact laptop with all the software and ideally the RACHEL image files, Kiwix ZIMs, etc. available for use. [I use an Leonvo X220, and/or a Macbook Air as they’re robust, powerful, light, and good quality computers.] If you intend to compile or develop software, prepare it with the extra software packages, compilers, etc. Update the software before heading to a location, etc. Remember to take the power supply and suitable power lead, including an adapter if needed for any of the locations you expect to visit.
  • Memory card readers (and writers): To read and write files, and Raspberry Pi images from a computer, using a USB port. Some computers include a memory card slot (typically SD-Card size), which are a reasonable alternative. Have at least one trustworthy adapter for micro SC-Cards.
  • Portable Hard disk(s) with copies of software we use. 500GB minimum capacity, 1TB recommended. Store copies of the packaged RACHEL distributions, and the unpacked RACHEL USB materials. Also include 7-zip installation files for Microsoft Windows, and various anti-virus programs.
  • USB memory sticks, currently I’d recommend carrying several 32GB and several 64GB drives. All USB 3.0 if you have a computer with at least one USB 3.0 port.
  • Cables, adapters, spares: At least one HDMI, one Ethernet, several micro-USB, cables, etc. A suitable HDMI-to-VGA cable (there are several types, check whether it’ll work with a Raspberry Pi before buying the cable. TBC - provide more details if/when I have them available.
  • Small USB Keyboard and portable display (ideally HDMI): Small 12v displays are sold globally to fit in cars. Some of these support VGA, most accept composite-Video (which Raspberry Pi’s provide), a tiny number (at least currently) support HDMI. The USB keyboard and display enable us to work directly on the small computer, and can be the decider between understanding and fixing a problem in the field vs. having to take the equipment off-site and work on them elsewhere. Note: many flatscren TV’s, including those in hotel rooms and and in bars (for watching the ever-popular Premier League football games) have HDMI connects and can be repurposed as very effective Raspberry Pi displays.
  • Wi-Fi hotspot: One of several ways to connect the devices together.
  • [Crossover] Ethernet cable: Another good, and minimalist way to connect Raspberry Pi Model B’s (and other devices with a working Ethernet port) to our laptop so we can manage and update them.
  • 3G router, local SIM card with data bundle: Useful for downloading updates, searching for answers on the web, sending information and requests for help, etc.
  • Portable loudspeaker: so people can hear audio in a larger area e.g. a classroom. The small rechargeable ones, such as the Wowee, are excellent.
  • Digital camera for pictures and recording videos:
  • Notebook: Highly portable and usable virtually anywhere (except when it’s dark) to make notes, draw diagrams, communicate some ideas and concepts, etc. Remember to have pens and pencils too.
  • Portable projector: Optional, could be used to teach or demonstrate materials to a group of people. I use one from Brookstone in the USA, which runs on 12v (so can be powered from a car battery) and gives 200 lumens of light - quite good for the size and weight.
  • Power adapters and power strips: even when somewhere has power, they seldom have enough power strips to keep your various equipment in working order. Travelling to other locations means you may also need power adapters for the mains sockets.
  • Solar panel & rechargable USB power bank(s): Useful for testing whether solar power is viable in a particular location. The USB powerbank can be charged (if the solar is viable) and used to power various devices, including a Raspberry Pi, an Android tablet, etc.
  • Anti-virus and virus-removal software: how else will you check and clean Windows computers you have to work on?
  • Rechargeable torch, possibly integrated in a USB powerpack such as the EasyAcc PowerBank U-Bright 3000 (which doubles as an ‘online’ battery).

Partition formats

Computer disks and drives need to be formatted before files can be stored on them. There are various common formats in use. Sometimes we need to choose the format carefully otherwise we may run out of space, sometimes unnecessarily, when there are large files (e.g. more than 2GB) and many files (as each file needs to be recorded in an invisible, finite table of files). * FAT32: The default format used on many drives when they are new. It has various limitations in terms of capacity of files. Not recommended for RACHEL files. * exFAT: Supported in Windows XP Service Pack 2 and later with an update from Microsoft. * NTFS: Default for Windows disks in computers. Not ideal for removable drives, files may be lost and drives corrupted unless you are both careful and lucky. * ext3: Linux only format. May work well for drives intended to be dedicated to Raspberry Pi’s and micro SD-cards dedicated to Android Tablets. Like NTFS, not really intended or suitable for removable storage.

Tools of the trade

Years of working with slow unreliable networks means we need ways to download files over the Internet efficiently and effectively even when timeouts and other problems occur. For instance, software that can continue a part-completed download, rather than restarting can make the difference between getting a job done and a sleepless night spent trying to retry the download tens of times.

Unix utilities such as curl or wget support resuming of downloads and tends to work better than trying to download a large file in a web browser. Take the destination URL from the web browser and give it to curl e.g. curl http://rachelfriends.org/rachelpi_64EN/rachelpi2_64EN_3.4.img --retry 999 --retry-max-time 0 -O -C - enabled me to download over 60GB over about 36 hours in Addis Ababa even though the network failed repeatedly.

For Windows computers, cygwin http://cygwin.com/install.html provides a basic set of linux commands and can be tailored to include virtually everything we might need from a development and systems administration perspective.

Travelling with paraphenalia

We need ways to travel practically with all the paraphenalia, and any other devices, etc. This is particularly important when flying and in areas where we are at risk or opportunistic theft. I travel with various bags, some clear, some stronger fabric bags with zips in bright colours, where I store the many and various pieces of equipment. For this workshop I have brought around 100 discrete pieces of equipment. Technology should always be carried in hand-luggage and be with you on the flight. Be careful to limit the combined weight and dimensions of the hand-baggage otherwise the airline may insist the bag goes in the hold (and then be at greater risk of theft at some airports in the continent).

Suggested Topics

Here’s the place to add any topics we’d like to suggest for this book.

20-May-2015: New topics asked at the workshop at elearning-africa * Can the Android apps work on BlackBerry 10 devices? Potentially yes, they should, if the relevant API’s are supported. However Kiwix uses a C++ library that might need some work to compile / port. And the small webserver used by Android RACHEL may not work either. * How to communicate, digitally, in nomadic environments in North Sudan. * RACHEL in Arabic?