Difference between revisions of "Building in Docker"

From DUNE
Jump to navigation Jump to search
 
(6 intermediate revisions by one other user not shown)
Line 1: Line 1:
 +
= ARCHIVAL =
 +
<hr/>
 +
<hr/>
 +
<hr/>
 +
<hr/>
 +
<hr/>
 +
<hr/>
 +
 
= Overview =
 
= Overview =
  
Line 5: Line 13:
 
<b>Note: This method is under development.</b>
 
<b>Note: This method is under development.</b>
  
−
== Theory ==
+
== Build Space ==
 +
 
 +
To put it in general terms,
 +
the problem being solved here is how to describe and populate a <b>build space</b>.  Here, a build space is a (binary) scalar field across a discrete, multi-dimensional space with the following axes:
 +
 
 +
; platforms : A <b>platform</b> is specified by its distribution name (Debian, Ubuntu, Fedora, Scientific, Mac OS X) its release version string and a set of additional OS-level packages.
 +
 
 +
; releases : A <b>release</b> is specified by a git tag, hash or other ref on {{gh|repo=lbne-build}}.
 +
 
 +
; environment : The <b>environment</b> includes any peculiarities about the environment in which the build is run which is independent from the above.
 +
 
 +
The binary scalar value of this space is then either "true" if its combination is supported for "false" if not.  This scalar field (and the defined extent of the space) changes over time as new platforms are created, new software releases are made, new build environments are added and their combined support is added or removed.
 +
 
 +
To be meaningful the <em>platform</em> actually has dependence on the <em>release</em>.  In general, different releases will make different demands on prerequisite packages provided by the platform.  The base distribution may or may not provide these and thus they must be met by including them in the set of additional OS-level packages.
 +
 
 +
The <em>environment</em> which is independent from <em>platform</em> and <em>releases</em> includes things like the need to account for HTTP/FTP proxies when building on a host present on a LAN which is firewalled from the Internet.  It can also include basic networking such as which DNS servers to use.  While these things are separate from the <em>platform</em> they imply a level of custom configuration of that platform which is dependent on the vagaries of the environment in which the build takes place.  Note that this environment constraint has implication at build time and later run-time of the build products.
 +
 
 +
== Goals ==
 +
 
 +
The goals in using Docker are:
 +
 
 +
* Provide experts a way to easily test releases on multiple platforms.
 +
* Precisely document/define what platforms are supported.
 +
* Provide a well defined basis for CI testing.
 +
* Provide an additional method for "casual" developers to access more platforms.
 +
 
 +
What are expclicitly <b>not</b> goals but are nonetheless potentially beneficialal side-effects are to distribute the resulting Docker images.  They may end up being useful for "turnkey" end-user running and development.
 +
 
 +
== Strategy ==
 +
 
 +
Note: currently this only applies to platforms that run Linux (ie, Mac OS X excluded).
 +
 
 +
At any given time, a subset of the space is described in a concise configuration file.  Each section of the file represents one "build point".  The section is interpreted by building a Docker image containing the platform, release and environment.  If the build and any subsequent validations succeed then the "binary scalar field" has a value of "true" at the corresponding point.
 +
 
 +
An attempt to handle all known environments is made by supplying shell initialization code that will "do the right thing" depending on what environment it encounters.
 +
 
 +
== Naming Conventions ==
 +
 
 +
Each point in the build space is named like:
 +
 
 +
<nowiki>lbne-<release>-<distroname>-<distroversion>-<build></nowiki>
 +
 
 +
Where:
  
−
This is a bit tongue-in-cheek but describes the problem in general terms.  
+
; lbne : just a marker
−
The problem being solved here is how to populate a (binary) scalar field across a discrete, multi-dimensional space with the following axes:
+
; release : an identifier for the LBNE software release version.
 +
; distroname : lower-case, one word names the GNU/Linux distribution ("debian", "fedora", "scientific", "ubuntu", etc)
 +
; distroversion : dotted-numerical released version string ("7.7", "14.04", etc)
 +
; build : a monotonically increasing integer to number any changes not captured by the other identifiers such as a change in additional OS-level packages needed to satisfy any prerequisites.
  
−
; platforms : Identified by a pair of (name,version) which gives the distribution name (Debian, Ubuntu, Fedora, Scientific, Mac OS X) plus its version.
+
Example:  
−
; releases : the release of the LBNE software and identified by a git tag on {{gh|repo=lbne-build}}.
 
−
; prerequisites : the list of software packages required by a given release but not provided as part of the base platform.  This list is largely constant but is weakly dependent on both the platform and the release.  Each "tick" on this axis is one unique set of packages and conceptually we ignore the fact that each distribution "spells" their names differently.
 
−
; environment : this includes peculiarities about the environment in which the build is run which is independent from the above.  In particular it includes any LAN-specific things like the need to account for explicit HTTP/FTP proxies
 
  
−
The binary scalar value of this space is then either "true" if its combination is supported for "false" if not. This scalar field and the extent of the space changes over time as new platforms are created, new software releases are made, etc, and their support is added or removed.
+
lbne-0.5.1-debian-7.7-1

Latest revision as of 17:32, 30 November 2017

ARCHIVAL

Overview

The Software Builds can be done with and in Docker containers using lbne-docker . See the README file there for details.

Note: This method is under development.

Build Space

To put it in general terms, the problem being solved here is how to describe and populate a build space. Here, a build space is a (binary) scalar field across a discrete, multi-dimensional space with the following axes:

platforms 
A platform is specified by its distribution name (Debian, Ubuntu, Fedora, Scientific, Mac OS X) its release version string and a set of additional OS-level packages.
releases 
A release is specified by a git tag, hash or other ref on lbne-build .
environment 
The environment includes any peculiarities about the environment in which the build is run which is independent from the above.

The binary scalar value of this space is then either "true" if its combination is supported for "false" if not. This scalar field (and the defined extent of the space) changes over time as new platforms are created, new software releases are made, new build environments are added and their combined support is added or removed.

To be meaningful the platform actually has dependence on the release. In general, different releases will make different demands on prerequisite packages provided by the platform. The base distribution may or may not provide these and thus they must be met by including them in the set of additional OS-level packages.

The environment which is independent from platform and releases includes things like the need to account for HTTP/FTP proxies when building on a host present on a LAN which is firewalled from the Internet. It can also include basic networking such as which DNS servers to use. While these things are separate from the platform they imply a level of custom configuration of that platform which is dependent on the vagaries of the environment in which the build takes place. Note that this environment constraint has implication at build time and later run-time of the build products.

Goals

The goals in using Docker are:

  • Provide experts a way to easily test releases on multiple platforms.
  • Precisely document/define what platforms are supported.
  • Provide a well defined basis for CI testing.
  • Provide an additional method for "casual" developers to access more platforms.

What are expclicitly not goals but are nonetheless potentially beneficialal side-effects are to distribute the resulting Docker images. They may end up being useful for "turnkey" end-user running and development.

Strategy

Note: currently this only applies to platforms that run Linux (ie, Mac OS X excluded).

At any given time, a subset of the space is described in a concise configuration file. Each section of the file represents one "build point". The section is interpreted by building a Docker image containing the platform, release and environment. If the build and any subsequent validations succeed then the "binary scalar field" has a value of "true" at the corresponding point.

An attempt to handle all known environments is made by supplying shell initialization code that will "do the right thing" depending on what environment it encounters.

Naming Conventions

Each point in the build space is named like: 

lbne-<release>-<distroname>-<distroversion>-<build>

Where:

lbne 
just a marker
release 
an identifier for the LBNE software release version.
distroname 
lower-case, one word names the GNU/Linux distribution ("debian", "fedora", "scientific", "ubuntu", etc)
distroversion 
dotted-numerical released version string ("7.7", "14.04", etc)
build 
a monotonically increasing integer to number any changes not captured by the other identifiers such as a change in additional OS-level packages needed to satisfy any prerequisites.

Example: 

lbne-0.5.1-debian-7.7-1
Retrieved from "https://dune.bnl.gov/w/index.php?title=Building_in_Docker&oldid=2163"