Difference between revisions of "Building in Docker"

From DUNE
Jump to navigation Jump to search
 
(2 intermediate revisions by one other user not shown)
Line 1: Line 1:
 +
= ARCHIVAL =
 +
<hr/>
 +
<hr/>
 +
<hr/>
 +
<hr/>
 +
<hr/>
 +
<hr/>
 +
 
= Overview =
 
= Overview =
  
Line 7: Line 15:
 
== Build Space ==
 
== Build Space ==
  
−
This is a bit tongue-in-cheek but describes the problem in general terms.
+
To put it in general terms,
−
+
the problem being solved here is how to describe and populate a <b>build space</b>.  Here, a build space is a (binary) scalar field across a discrete, multi-dimensional space with the following axes:
−
The problem being solved here is how to describe and populate a <b>build space</b>.  Here, a build space is a (binary) scalar field across a discrete, multi-dimensional space with the following axes:
 
  
 
; platforms : A <b>platform</b> is specified by its distribution name (Debian, Ubuntu, Fedora, Scientific, Mac OS X) its release version string and a set of additional OS-level packages.
 
; platforms : A <b>platform</b> is specified by its distribution name (Debian, Ubuntu, Fedora, Scientific, Mac OS X) its release version string and a set of additional OS-level packages.
Line 22: Line 29:
  
 
The <em>environment</em> which is independent from <em>platform</em> and <em>releases</em> includes things like the need to account for HTTP/FTP proxies when building on a host present on a LAN which is firewalled from the Internet.  It can also include basic networking such as which DNS servers to use.  While these things are separate from the <em>platform</em> they imply a level of custom configuration of that platform which is dependent on the vagaries of the environment in which the build takes place.  Note that this environment constraint has implication at build time and later run-time of the build products.
 
The <em>environment</em> which is independent from <em>platform</em> and <em>releases</em> includes things like the need to account for HTTP/FTP proxies when building on a host present on a LAN which is firewalled from the Internet.  It can also include basic networking such as which DNS servers to use.  While these things are separate from the <em>platform</em> they imply a level of custom configuration of that platform which is dependent on the vagaries of the environment in which the build takes place.  Note that this environment constraint has implication at build time and later run-time of the build products.
 +
 +
== Goals ==
 +
 +
The goals in using Docker are:
 +
 +
* Provide experts a way to easily test releases on multiple platforms.
 +
* Precisely document/define what platforms are supported.
 +
* Provide a well defined basis for CI testing.
 +
* Provide an additional method for "casual" developers to access more platforms.
 +
 +
What are expclicitly <b>not</b> goals but are nonetheless potentially beneficialal side-effects are to distribute the resulting Docker images.  They may end up being useful for "turnkey" end-user running and development.
  
 
== Strategy ==
 
== Strategy ==
Line 29: Line 47:
 
At any given time, a subset of the space is described in a concise configuration file.  Each section of the file represents one "build point".  The section is interpreted by building a Docker image containing the platform, release and environment.  If the build and any subsequent validations succeed then the "binary scalar field" has a value of "true" at the corresponding point.
 
At any given time, a subset of the space is described in a concise configuration file.  Each section of the file represents one "build point".  The section is interpreted by building a Docker image containing the platform, release and environment.  If the build and any subsequent validations succeed then the "binary scalar field" has a value of "true" at the corresponding point.
  
−
A partial build consisting of just the platform is done as it is expected to be (re)used by more than one release.  An attempt to handle all known environments is made by supplying shell initialization code that will "do the right thing" depending on what environment it encounters.
+
An attempt to handle all known environments is made by supplying shell initialization code that will "do the right thing" depending on what environment it encounters.
  
 
== Naming Conventions ==
 
== Naming Conventions ==
Line 35: Line 53:
 
Each point in the build space is named like:
 
Each point in the build space is named like:
  
−
  <nowiki><distroname>-<distroversion>-<packagenum>[-<release>]</nowiki>
+
  <nowiki>lbne-<release>-<distroname>-<distroversion>-<build></nowiki>
  
 
Where:
 
Where:
  
−
; distroname : names the GNU/Linux distribution (Debian, etc)
+
; lbne : just a marker
−
; distroversion : names the released version
+
; release : an identifier for the LBNE software release version.
−
; packagenum : a simple monotonically increasing integer that counts changes in the set of additional OS-level packages installed
+
; distroname : lower-case, one word names the GNU/Linux distribution ("debian", "fedora", "scientific", "ubuntu", etc)
−
; release : an identifier for the LBNE software release. This is not specified for builds that are meant to only include the platform
+
; distroversion : dotted-numerical released version string ("7.7", "14.04", etc)
 +
; build : a monotonically increasing integer to number any changes not captured by the other identifiers such as a change in additional OS-level packages needed to satisfy any prerequisites.
 +
 
 +
Example:  
 +
 
 +
lbne-0.5.1-debian-7.7-1

Latest revision as of 17:32, 30 November 2017

ARCHIVAL

Overview

The Software Builds can be done with and in Docker containers using lbne-docker . See the README file there for details.

Note: This method is under development.

Build Space

To put it in general terms, the problem being solved here is how to describe and populate a build space. Here, a build space is a (binary) scalar field across a discrete, multi-dimensional space with the following axes:

platforms 
A platform is specified by its distribution name (Debian, Ubuntu, Fedora, Scientific, Mac OS X) its release version string and a set of additional OS-level packages.
releases 
A release is specified by a git tag, hash or other ref on lbne-build .
environment 
The environment includes any peculiarities about the environment in which the build is run which is independent from the above.

The binary scalar value of this space is then either "true" if its combination is supported for "false" if not. This scalar field (and the defined extent of the space) changes over time as new platforms are created, new software releases are made, new build environments are added and their combined support is added or removed.

To be meaningful the platform actually has dependence on the release. In general, different releases will make different demands on prerequisite packages provided by the platform. The base distribution may or may not provide these and thus they must be met by including them in the set of additional OS-level packages.

The environment which is independent from platform and releases includes things like the need to account for HTTP/FTP proxies when building on a host present on a LAN which is firewalled from the Internet. It can also include basic networking such as which DNS servers to use. While these things are separate from the platform they imply a level of custom configuration of that platform which is dependent on the vagaries of the environment in which the build takes place. Note that this environment constraint has implication at build time and later run-time of the build products.

Goals

The goals in using Docker are:

  • Provide experts a way to easily test releases on multiple platforms.
  • Precisely document/define what platforms are supported.
  • Provide a well defined basis for CI testing.
  • Provide an additional method for "casual" developers to access more platforms.

What are expclicitly not goals but are nonetheless potentially beneficialal side-effects are to distribute the resulting Docker images. They may end up being useful for "turnkey" end-user running and development.

Strategy

Note: currently this only applies to platforms that run Linux (ie, Mac OS X excluded).

At any given time, a subset of the space is described in a concise configuration file. Each section of the file represents one "build point". The section is interpreted by building a Docker image containing the platform, release and environment. If the build and any subsequent validations succeed then the "binary scalar field" has a value of "true" at the corresponding point.

An attempt to handle all known environments is made by supplying shell initialization code that will "do the right thing" depending on what environment it encounters.

Naming Conventions

Each point in the build space is named like: 

lbne-<release>-<distroname>-<distroversion>-<build>

Where:

lbne 
just a marker
release 
an identifier for the LBNE software release version.
distroname 
lower-case, one word names the GNU/Linux distribution ("debian", "fedora", "scientific", "ubuntu", etc)
distroversion 
dotted-numerical released version string ("7.7", "14.04", etc)
build 
a monotonically increasing integer to number any changes not captured by the other identifiers such as a change in additional OS-level packages needed to satisfy any prerequisites.

Example: 

lbne-0.5.1-debian-7.7-1
Retrieved from "https://dune.bnl.gov/w/index.php?title=Building_in_Docker&oldid=2163"