Posts on widerin

Gitlab Security Products with Java and Gradle

Sun, 08 Mar 2020 00:00:00 +0000

GitLab security products have been constantly improving support for different build tools and programming languages.

My recent contribution to License compliance allows now to inject commands SETUP_CMD environment variable to the core to bypass the package manager auto-detection and make use of the ./gradlew Gradle wrapper script.

For example:

include:
- template: Security/License-Scanning.gitlab-ci.yml
variables:
SETUP_CMD: sh my-custom-install-script.sh

In this example, my-custom-install-script.sh is a shell script at the root directory of your project.

Java versions on macOS

Fri, 21 Feb 2020 00:00:00 +0000

On macOS you can use /usr/libexec/java_home command to control your used Java version. As a lot of applications and/or projects still require Java 1.8 or you might want to use an alternative JDK/JRE let’s look into managing those different versions.

$ /usr/libexec/java_home -h
Usage: java_home [options...]
Returns the path to a Java home directory from the current user's settings.
Options:
[-v/--version <version>] Filter Java versions in the "JVMVersion" form 1.X(+ or *).
[-a/--arch <architecture>] Filter JVMs matching architecture (i386, x86_64, etc).
[-d/--datamodel <datamodel>] Filter JVMs capable of -d32 or -d64
[-t/--task <task>] Use the JVM list for a specific task (Applets, WebStart, BundledApp, JNI, or CommandLine)
[-F/--failfast] Fail when filters return no JVMs, do not continue with default.
[ --exec <command> ...] Execute the $JAVA_HOME/bin/<command> with the remaining arguments.
[-R/--request] Request installation of a Java Runtime if not installed.
[-X/--xml] Print full JVM list and additional data as XML plist.
[-V/--verbose] Print full JVM list with architectures.
[-h/--help] This usage information.

Manual installation

For this post I’ve decided to use AdoptOpenJDK as it’s actively maintained.

Gitlab License Management with Java 11

Fri, 21 Dec 2018 00:00:00 +0000

If built-in License Management functionality does not support the programming language requirements you have, you can customize the License Management docker image.

This guide shows you how to use License Management with OpenJDK 11 or later.

License Management definition

License Management helps you find what licenses your project uses in its dependencies and decide for each of then whether to allow it or forbid it.

License Compliance report under pipelines

GitLab SAST and Java 11

Wed, 19 Dec 2018 00:00:00 +0000

If built-in Static Application Security Testing functionality does not support the programming language requirements you have, you can customize the analyzer docker images in [GitLab]’s sast pipeline job.

This guide shows you how to use SAST with OpenJDK 11 or later.

SAST definition

Static Application Security Testing provides you information if

your application is vulnerable by using an external library in a specific version which is known to be vulnerable or
your code has a potentially dangerous attribute in a class, or unsafe code that can lead to unintended code execution.

Project Security Dashboard

How to run Mikrotik's Winbox on macOS

Wed, 24 Jan 2018 00:00:00 +0000

Some time ago m0n0wall burried their project and my firewall went out of maintenance. Back in 2008 m0n0wall really had its great time but unfortunately the recommended successor project opnsense required more hardware (cpu/memory) than my alix routerboard alix2c3 was equipped with.

I didn’t want to invest or assemble a new alix routerboard with encosure and antennas. So after doing some research online I decided Mikrotik would be my next internet entry point.

So I got a hAP AC from Mikrotik.

Official Grafana docker image on OpenShift 3.x

Sun, 10 Sep 2017 00:00:00 +0000

Running offical docker images from Docker hub on OpenShift can be painful as it uses arbitrary user ids by default. Often you end up building your own or using 3rd party forks/clones of the offical image, which leds to additional effort to keep it up to date in terms of security and/or version upgrades.

Background

Technically speaking, [OpenShift] generates a dynamic uid to launch a container which will not have an associated entry in the containers /etc/passwd file. At docker image build time this user does not exist so chown files and/or directories to this uid is not possible.

Grafana’s docker image

gosu

Grafana runs its service as grafana user and uses an entrypoint script which runs as root and launches grafana-server as grafana user by using gosu, see run.sh on github.com

Hashing algorithms in Python 3

Wed, 17 May 2017 00:00:00 +0000

While working on a larger project there was a need to detect some changes happened in given data structures. Usually, you immediately start over using the default hashing algorithm md5:

Python 3.4.5 (default, Jan 14 2017, 22:06:30)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

Let’s compare 2 strings:

>>> aa = "Hello World"
>>> bb = "Hello World"
>>> aa == bb
True

So obviouly, these strings are equal. So also hashlib should confirm this:

Continuous integration with GitLab and Docker

Tue, 11 Oct 2016 00:00:00 +0000

I’ve given a talk about continuous integration with GitLab and Docker on Developer Meetup VlbgWebDev. The Meetup takes place every second Tuesday every Month and i’ve met some old colleagues there.

It was a nice Meetup with lots of discussions, that’s why i published my sample code to GitLab and my slides to Speakerdeck. First intention was to do some Pyramid demo but after I’ve heard there are some Flask people, I’ve added Flask demos as well :-)

Weblate and i18next json files

Fri, 29 Apr 2016 00:00:00 +0000

There’s a bug report on github about JSON format breaks i18next files due to automatic restructuring. Actually it flattens your i18next JSON file and makes it unusable for further usage.

So after starting with a JSON file like:

{
"navbar": {
"reward": "Rewards",
"about": "About"
}
}

you end up with a committed file which was transformed into this:

{
"navbar.reward": "Rewards",
"navbar.about": "About"
}

Use Weblate PRE_COMMIT_SCRIPTS

Weblate offers script hooks before or after a repository action happens. I’m using one of those hooks to execute a short Python script which reformats JSON again to be usable in i18next translation tools again. The script below is also available on Github:

GitLab as OAuth provider for Weblate

Thu, 28 Apr 2016 00:00:00 +0000

Weblate is built on Django and you can use a lot of authentication plugins. After my contribution to django-allauth and the backporting work of glensc and omab to django-social-auth it’s very easy to integrate [GitLab]’s authentication provider with Weblate

GitLab

All you have to do on GitLab is configuration through webinterface. Go to https://gitlab.com/profile/applications and register a new application.

Name: Weblate
Redirect URI: `https://weblate.yourdomain.tld/accounts/complete/GitLab/``

After registering your new application you’ll get your Application Key and Secret. You’ll need this later for configuration

Change umask in Docker containers

Mon, 25 Jan 2016 00:00:00 +0000

Problem

You’re mounting volumes into Docker containers and running into permission issues while accessing volume from other Docker containers.

Docker installation required

Docker must be installed on your system.

$ docker version
Client: Docker Engine - Community
 Version: 19.03.5
 API version: 1.40
 Go version: go1.12.12
 Git commit: 633a0ea
 Built: Wed Nov 13 07:22:34 2019
 OS/Arch: darwin/amd64
 Experimental: false

Server: Docker Engine - Community
 Engine:
 Version: 19.03.5
 API version: 1.40 (minimum version 1.12)
 Go version: go1.12.12
 Git commit: 633a0ea
 Built: Wed Nov 13 07:29:19 2019
 OS/Arch: linux/amd64
 Experimental: false
 containerd:
 Version: v1.2.10
 GitCommit: b34a5c8af56e510852c35414db4c1f4fa6172339
 runc:
 Version: 1.0.0-rc8+dev
 GitCommit: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
 docker-init:
 Version: 0.18.0
 GitCommit: fec3683

umask

umask is set to 0022 on most linux sytems by default. This umask allows group and others to read but not write your files, see below:

Connect devpi to your docker container

Thu, 07 Jan 2016 00:00:00 +0000

You’re running pip install, buildout or your automated tests within a docker container and after running it multiple times you’ve become tired of waiting for it to finally complete? Then it’s time to connect a devpi server.

Docker installation required

Docker must be installed on your system.

$ docker version
Client: Docker Engine - Community
 Version: 19.03.5
 API version: 1.40
 Go version: go1.12.12
 Git commit: 633a0ea
 Built: Wed Nov 13 07:22:34 2019
 OS/Arch: darwin/amd64
 Experimental: false

Server: Docker Engine - Community
 Engine:
 Version: 19.03.5
 API version: 1.40 (minimum version 1.12)
 Go version: go1.12.12
 Git commit: 633a0ea
 Built: Wed Nov 13 07:29:19 2019
 OS/Arch: linux/amd64
 Experimental: false
 containerd:
 Version: v1.2.10
 GitCommit: b34a5c8af56e510852c35414db4c1f4fa6172339
 runc:
 Version: 1.0.0-rc8+dev
 GitCommit: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
 docker-init:
 Version: 0.18.0
 GitCommit: fec3683

You’ll need a working directory where docker-compose.yml file and certificates are stored in, so please create a folder my-devpi as shown below:

Private docker-registry within docker

Thu, 07 Jan 2016 00:00:00 +0000

Sometimes you need your own docker-registry to just try out new things or to avoid downloading stuff again and again from the internet. I tried to build a docker-compose recipe to setup my own private registry.

Docker installation required

Docker must be installed on your system.

$ docker version
Client: Docker Engine - Community
 Version: 19.03.5
 API version: 1.40
 Go version: go1.12.12
 Git commit: 633a0ea
 Built: Wed Nov 13 07:22:34 2019
 OS/Arch: darwin/amd64
 Experimental: false

Server: Docker Engine - Community
 Engine:
 Version: 19.03.5
 API version: 1.40 (minimum version 1.12)
 Go version: go1.12.12
 Git commit: 633a0ea
 Built: Wed Nov 13 07:29:19 2019
 OS/Arch: linux/amd64
 Experimental: false
 containerd:
 Version: v1.2.10
 GitCommit: b34a5c8af56e510852c35414db4c1f4fa6172339
 runc:
 Version: 1.0.0-rc8+dev
 GitCommit: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
 docker-init:
 Version: 0.18.0
 GitCommit: fec3683

You’ll need a working directory where docker-compose.yml file and certificates are stored in, so please create a folder my-registry as shown below:

Maintaining manual gitlab installation with ansible

Sun, 03 Jan 2016 00:00:00 +0000

A long time ago we started using GitLab. It was a greate step forward and definitly the right decision to switch from SVN to a distributed version control system. If you want to try GitLab you should definitly install it using GitLab Omnibus distribution.

Manual installation

But this long time ago there was no omnibus installation available so we installed GitLab with a manual git checkout. Their monthly release cycle brought some upgrade effort every month - there was an upgrade tool available which was marked as discontinued - so i started to write some anisble scripts to handle the upgrade process.

Plone 6.0: What I expect

Tue, 31 Mar 2015 00:00:00 +0000

We’re still working on Plone 5.0 but for PLOG 2015 I put all my thoughts together and want to share with you what I expect Plone 6.0 to include.

Legacy, redundancy and why forcing a style guide

Plone has a lot of legacy code and this results in several different ways of doing things. Some developers prefer the old style some try to figure out a new way. Since we had no style guides in the past (we’re working hard on that) it was up to the developers themselves how to implement a piece of code.

Using SVN in eclipse on Mac OS X

Mon, 03 Feb 2014 00:00:00 +0000

After upgrading subversion client my eclipse could not find javahl bindings any more, because i previously installed the CollabNet ones, which are currently available for subversion 1.6.x only. The bad thing about this is Subclipse does not work any more, until you’ll install these libraries using the following solutions.

There may be one for macports as well, but i’m using homebrew and nothing else any more :-)

Installing through homebrew is not as easy as installing through macports because they are not available as single package, but you can add a --java option to the homebrew command to build required javahlbinding together with subversion. This will show up the following warning, because you should use a universal build.

Image filesize validator for dexterity content types in Plone

Fri, 31 May 2013 00:00:00 +0000

When using Plone 4.3 with plone.app.contenttypes you might want to limit max filesize for images by implementing a custom z3c.form validator. In ATCT we were able to limit max image filesize by overwriting ATCT config file or setting in portal_atct tool.

Using webserver

The simplest way to achieve a max-upload size is to just limit client_max_body_size in nginx webserver (or simliar in apache).

But my current solution is really different. It creates a validator for a special fieldtype, in our special case a INamedBlobImageField.

Use Babel to translate your python package

Mon, 01 Apr 2013 00:00:00 +0000

I use egg-containing buildouts for all my Plone packages. Since i18ndude does not extract msgids from zcml files, i tried Babel and succeeded.

Problem

You want to translate titles in Plone’s diplay menu by adding browser:menuItem in your zcml file as shown here.

<configure
xmlns="http://namespaces.zope.org/zope"
xmlns:browser="http://namespaces.zope.org/browser"
xmlns:plone="http://namespaces.plone.org/plone"
xmlns:i18n="http://namespaces.zope.org/i18n"
i18n:domain="my.package">
<browser:page
for="plone.folder.interfaces.IOrderableFolder"
name="a_new_view"
class=".demo_view.DemoView"
permission="zope2.View"
template="templates/demo_view.pt"
/>
<!-- Entry in display menu -->
<browser:menuItem
for="plone.folder.interfaces.IOrderableFolder"
menu="plone_displayviews"
title="A new view"
action="@@a_new_view"
description="I want a view with translated title and description"
i18n:attributes="title; description"
/>
</configure>

i18ndude does not extract message ids from zcml files, so i tried a diffent solution using Babel.

Turbo twitter Bootstrap theming for Plone using Diazo

Tue, 11 Dec 2012 00:00:00 +0000

When building a new Plone theme based on Twitter Bootstrap without modifying core templates you have to do a lot of xsl-styling. Hopefully this article can help you getting started.

Building a diazo theme

You should have basic knowledge of how to build a diazo theme in Plone. So you will have a file called rules.xml and hopefully your Twitter Bootstrap javascript and css code already plugged into Plone resource registries. An installable Plone product is available here.

Plone test development with robotframework and RIDE

Thu, 29 Nov 2012 00:00:00 +0000

Running acceptance tests in Plone

Test driven development (TDD) can be a pain. If you are working with Zope and Plone most of time gets consumed starting up Zope running setUp method and rolling back transaction in tearDown method.

So why not use a running already set-up instance for test-development? Today I will show you how to reach that goal. It’s not as hard as it seems.

Use buildout

Of course we use buildout to get a custom script which we can run through RIDE. I configured a new pybot script, the basic idea has been implemented in plone.act¹ module already.

Howto use buildout version pins?

Wed, 14 Nov 2012 00:00:00 +0000

When using Plone with buildout you want to be able to reproduce your environment with the same set of eggs, even if there were some new releases in between.

buildout 2.x

For buildout 2.x they implemented a lot of improvements for pinning versions.

allow-picked-versions allow buildout to download pinned packages only. Not pinned but required packages will raise an Exception.
show-picked-versions shows a summary of picked versions after your first buildout run. You can copy'n'paste this into a versions.cfg file.

buildout 1.x

Manually

Without pinning your eggs it will be impossible to reproduce the deployed buildout configuration in future. Use a this buildout extension for future projects to avoid this pitfall:

Install Robot Framework RIDE through homebrew

Tue, 16 Oct 2012 00:00:00 +0000

Python, wxPython and robotframework RIDE on your macOS.

Update 2016, 23 Feb

Fixed wxPython installation so it works with most recent versions available today.

Update 2017, 03 Jul

Seems this installation method no longer works on >= MacOS Sierra. Please use a docker image and Ride in browser.

wxPython

Since wxPython is available on homebrew you don’t have to install the package provided on the wxPython website.

$ brew info wxpython
wxPython: stable 3.0.2.0 (bottled)
Python bindings for wxWidgets
https://www.wxwidgets.org/
/usr/local/Cellar/wxPython/3.0.2.0 (1,107 files, 38.2M)
Poured from bottle
From: https://github.com/Homebrew/homebrew/blob/master/Library/Formula/wxpython.rb
==> Dependencies
Required: wxmac ✔
==> Options
--universal
$ brew install wxpython

This will take some time so be patient.

Secure your Dropbox

Wed, 10 Oct 2012 00:00:00 +0000

Quick guide howto get an encFs encrypted Dropbox subfolder automounted on login in macOS and indexed in Spotlight.

Introduction

First of all, Dropbox is a really useful piece of software. Your data is stored unencrypted inside an online storage and in this case it’s not suitable for secret data. You can bypass this disadvantage by encrypting a folder inside Dropbox.

Prerequisits

The following instructions expect you using a macOS but may also inspire Unix or Linux users.

Change minute_step in Archetypes DateTimeWidget

Tue, 31 Aug 2010 00:00:00 +0000

Sometimes you need minutes which could not be divided by 5 which Plone doesn’t select you by default. You don’t need to rewrite core components in Archetypes DateTimeWidget to change this.

Prerequisites

You need your own BrowserLayer (a skin or theme product) or a contenttype with at least one marker interface to customize on.

Howto

Create a file called datecomponents.py in your browser directory with following content:

from plone.app.form.widgets import datecomponents
class DateComponents(datecomponents.DateComponents):
def result(self, date=None, use_ampm=False, starting_year=None,
ending_year=None, future_years=None, minute_step=5):
# Change minute steps to 1
return super(DateComponents, self).result(date, use_ampm,
starting_year, ending_year,
future_years, 1)

… and register it for you browserlayer (replace IThemeSpecific by your layer interface) or for a type you want to customize its widget (replace for='*' with your types interface), thats it.

Howto send mails from localhost through a smarthost on macOS

Wed, 07 Apr 2010 00:00:00 +0000

You’re getting weird errors like “Connection refused” from your smtp library? By reconfiguring your local mailer to send mails through a smarthost you can solve this issue.

Setting up a smarthost

Setting up your macOS to use a smarthost to forward mails to is not very hard. I found a nice tutorial on fuerstnet.de which was the base of this blog post.

Copy the lines:

$ sudo postconf -e relayhost=your.isps.mailserver smtp_use_tls=yes smtp_sasl_auth_enable=yes smtp_sasl_password_maps=hash:/etc/postfix/smtp_auth tls_random_source=dev:/dev/urandom smtp_sasl_security_options=noanonymous
$ sudo sh -c 'echo "your.isps.mailserver username:password" >> /etc/postfix/smtp_auth'
$ sudo chown root:wheel /etc/postfix/smtp_auth
$ sudo chmod 600 /etc/postfix/smtp_auth
$ sudo postmap hash:/etc/postfix/smtp_auth

Problems

I noticed postfix is quitting after 60 seconds. So if you send yourself a test email using: echo "test" | mail -s "test" your@email the postfix daemon will quit after 60 seconds and you’ll get these errors when trying to send mails through localhost in (for example) Plone:

Tell svn client to use a proxy-server

Tue, 09 Mar 2010 00:00:00 +0000

If you have or your server has to use a proxy server to access the internet you may have problems with your svn client although you aready set your http_proxy environment variable(s).

The solution to tell svn to use a proxy server is editing the ~/.subversion/servers file in your home directory and add the following lines or uncomment them, if they are already written into it:

[global]
# http-proxy-exceptions = *.exception.com, www.internal-site.org
http-proxy-host = my.proxy.server
http-proxy-port = 8080
# http-proxy-username = defaultusername
# http-proxy-password = defaultpassword
# http-compression = no
# http-auth-types = basic;digest;negotiate
# No http-timeout, so just use the builtin default.
# No neon-debug-mask, so neon debugging is disabled.
# ssl-authority-files = /path/to/CAcert.pem;/path/to/CAcert2.pem

Howto use custom color for collective.captcha?

Thu, 23 Apr 2009 00:00:00 +0000

How to use your custom color with collective.captcha and Plone? This tutorial shows you an easy way to customize it using your own BrowserLayer.

collective.captcha uses a BrowserView to display it’s captcha image. Therefore it’s very easy to customize it for your own theme because you can subclass that BrowserView for your own theme layer which is a BrowserLayer. You can change colors and/or fonts as you need it for your custom theme without doing any code change on the original collective.captcha package.