Docker II: Quick Airflow

FROM python:3.7
RUN pip3 install 'apache-airflow' 
RUN airflow initdb 
CMD (airflow scheduler &) && airflow webserver 

Docker commands

# Start container
docker build . -t airflow
docker run -it -p 8080:8080 -v :/root/airflow airflow
# Save it go DockerHub
docker commit cxxxxxxxxx cadvga/airflow-version1
docker push cadvga/airflow-version1

Docker basic CheatSheet

  • Install Docker:
  • Test Docker: docker run hello-world
  • Help:
    • docker –version
    • docker –help
    • docker COMMAND –help
  • Commands:
    • Show available images: docker images
    • Show containers: docker ps (docker container ls)
    • Manage containers:
      • List: docker container ls
      • Stop: docker container stop dsa34ewqe
      • Force stop: docker container kill dsa34ewqe
      • Remove: docker container rm dsa34ewqe
      • Execute bash: docker exec -it dsa34ewqe bash
      • Remove
    • Manage images:
      • List image: docker image ls
      • Remove image: docker rmi ubuntu/latest
      • Remove all images: docker rmi $(docker images -a -q)
      • Dangling images:
        • List: docker images -f dangling=true
        • Delete: docker images purge
      • Create image: docker build -t friendlyhello .
      • Run image:
        • docker run -d -p 4000:80 friendlyhello
        • docker run -d -p 4000:80 cadvga/get-started:part2
    • Manage compose:
      • docker-compose up -d

Elastic Search API quick reference guide

Index APIs

  • Create index: PUT /index_name
    • Can have JSON body to specify: Settings, Mappings, Aliases.
  • Check index existence: HEAD /index_name
  • Get index information: GET /index_name
  • Get index documents: GET /index_name/_search
  • Get index mappings: GET /index_name/_mappings
  • Delete index: DELETE /index_name
  • Delete documents: POST /index_name/_delete_by_query
  • Get settings: GET /{index}/_settings
  • Update settings: PUT /{index}/_settings

Cluster APIs

  • Cluster Health:
    • GET /_cluster/health
    • GET /_cluster/health/idx1,idx2 (Specified indices)
  • Cluster stats: GET /_cluster/stats
  • Cluster state: GET /_cluster/state
  • Nodes statistics: GET /_nodes/stats
  • Nodes info: GET /_nodes

Cat APIs

  • Command: GET /_cat/command_name

    • v: turn on verbose (header) output.
    • help: show available columns.
    • h: limit the output columns.
    • … (see cat APIs) for more details.
  • List of all command with: GET /_cat

Useful slang to communicate with data scientists

  • Definition: A precise and unambiguous description of the meaning of a mathematical term. (Everybody knows that…)
  • Claim:Assertion that is then proved.  It is often used like an informal lemma. (I say…)
  • Axiom/Postulate: A statement that is assumed to be true without proof. (Believe me when I say…)
  • Lemma: A lemma is a useful observation that serves as a stepping stone to understand theorems. (I see…)
  • Conjecture: A conjecture is a guess mostly based on observations. (I guess…)
  • Hypothesis: A hypothesis is a conjecture with lots of partial evidences and significant consequences. (I believe…)
  • Theorem: A theorem is a single essential contribution to a theory (I know…)
  • Corollary: A result in which the (usually short) proof relies heavily on a given (ergo…)
  • Theory: This is how it works, based on this theorems (It is…)
  • Paradox: A statement that can be shown, using a given set of axioms and definitions, to be both true and false (WTF!)
  • […]
  • Grand Unified Theory of Everything: The ÜBER THEORY. (Oh my God… it is full of stars. Look…a Higgs Boson!)

Git-Flow alternative to support multiple versions of a product

Git-Flow works very well for a scenario where devs mantain & evolve a software with a single instance on production (i.e. git

But imagine a scenario where devs develop a software that has different versions installed in different clients and devs have to give some support to old versions. (i.e. Visual Studio) while evolving the latest. Devs have to:

  • Mantain version 1.0 of your software (critical bug fixing) [In our example VS 2013]
  • Mantain version 1.1 of your software (critical bug fixing) [In our example VS 2015]
  • Develop version 1.2 (new features) [In our example VS 2017]

Git-Flow doesn’t support that scenario. Here is my solution, a git flow variant with feature branches and multiple “Master” branches called Release Branches.


  • Feature Branches and Dev branch works like git-flow.
  • Version 1.0
    • First release (r1.0) is done by creating a branch.
    • Evolutive development continues in Dev branch (and feature branches)
    • It is critical that no evolutive development is done in Release Branch r1.0 so if user wants a new feature he has to wait to r1.1 version.
    • If a critical bug is found in r1.0 version it can be fixed there.
    • If the critical bug is affecting dev, bug fix must be merged to dev branch.
    • If the critical bug is complex a small hot-fix branch can be created (like a feature branch)
  • Version 1.1
    • When next version (r1.1) is ready a new branch is created [Same as previous version]
    • Old branch (r1.0) is kept while we have still users on that version.
    • Once version 1.0 (r1.0) is declared obsolete, users with bugs must install a higher version.
  • Version 1.2
    • Same as version 1.1
  • Etc.

Function to prepare Machine.config

This PowerShell script “prepare” machine.config by:

  • Create appSettings.config and link it in machine.config
  • Create connectionStrings.config and link it in machine.config
  • Add proxy settings








Hello world!

Welcome to WordPress. This is your first post. Edit or delete it, then start writing!

Main base Powershell script (with Azure)

Example of PowerShell Main base script with detection of PowerShell version and Azure PowerShell version.



PowerShell new features.

PowerShell 5.0 (Windows Management Framework 5.0)

  • By default in Windows 10 & Windows Server 2016 Technical Preview.
  • Remote Script Debugging in ISE 🙂
  • Transcription works in ISE by default
  • .NET enumerations.
  • Improved DSC.

PowerShell 4.0

  • By default in Windows 8.1 & Windows Server 2012 R2
  • DSC: Desired State Configuration
  • Remote Script Debugging in console (no ISE)
  • Improved Invoke-RestMethod and Invoke-WebRequest to specify headers.

PowerShell 3.0

  • By default in Windows 8 & Windows Server 2012
  • Improved ISE: IntelliSense
  • Improved remote PowerShell ( New-PSSession)
  • Module Auto-Load
  • Invoke-RestMethod and Invoke-WebRequest
  • Ordered Hashtables
  • Scheduled jobs