Development on widerin

Gitlab Security Products with Java and Gradle

Sun, 08 Mar 2020 00:00:00 +0000

GitLab security products have been constantly improving support for different build tools and programming languages.

My recent contribution to License compliance allows now to inject commands SETUP_CMD environment variable to the core to bypass the package manager auto-detection and make use of the ./gradlew Gradle wrapper script.

For example:

include:
- template: Security/License-Scanning.gitlab-ci.yml
variables:
SETUP_CMD: sh my-custom-install-script.sh

In this example, my-custom-install-script.sh is a shell script at the root directory of your project.

Java versions on macOS

Fri, 21 Feb 2020 00:00:00 +0000

On macOS you can use /usr/libexec/java_home command to control your used Java version. As a lot of applications and/or projects still require Java 1.8 or you might want to use an alternative JDK/JRE let’s look into managing those different versions.

$ /usr/libexec/java_home -h
Usage: java_home [options...]
Returns the path to a Java home directory from the current user's settings.
Options:
[-v/--version <version>] Filter Java versions in the "JVMVersion" form 1.X(+ or *).
[-a/--arch <architecture>] Filter JVMs matching architecture (i386, x86_64, etc).
[-d/--datamodel <datamodel>] Filter JVMs capable of -d32 or -d64
[-t/--task <task>] Use the JVM list for a specific task (Applets, WebStart, BundledApp, JNI, or CommandLine)
[-F/--failfast] Fail when filters return no JVMs, do not continue with default.
[ --exec <command> ...] Execute the $JAVA_HOME/bin/<command> with the remaining arguments.
[-R/--request] Request installation of a Java Runtime if not installed.
[-X/--xml] Print full JVM list and additional data as XML plist.
[-V/--verbose] Print full JVM list with architectures.
[-h/--help] This usage information.

Manual installation

For this post I’ve decided to use AdoptOpenJDK as it’s actively maintained.

Gitlab License Management with Java 11

Fri, 21 Dec 2018 00:00:00 +0000

If built-in License Management functionality does not support the programming language requirements you have, you can customize the License Management docker image.

This guide shows you how to use License Management with OpenJDK 11 or later.

License Management definition

License Management helps you find what licenses your project uses in its dependencies and decide for each of then whether to allow it or forbid it.

License Compliance report under pipelines

GitLab SAST and Java 11

Wed, 19 Dec 2018 00:00:00 +0000

If built-in Static Application Security Testing functionality does not support the programming language requirements you have, you can customize the analyzer docker images in [GitLab]’s sast pipeline job.

This guide shows you how to use SAST with OpenJDK 11 or later.

SAST definition

Static Application Security Testing provides you information if

your application is vulnerable by using an external library in a specific version which is known to be vulnerable or
your code has a potentially dangerous attribute in a class, or unsafe code that can lead to unintended code execution.

Project Security Dashboard

Hashing algorithms in Python 3

Wed, 17 May 2017 00:00:00 +0000

While working on a larger project there was a need to detect some changes happened in given data structures. Usually, you immediately start over using the default hashing algorithm md5:

Python 3.4.5 (default, Jan 14 2017, 22:06:30)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

Let’s compare 2 strings:

>>> aa = "Hello World"
>>> bb = "Hello World"
>>> aa == bb
True

So obviouly, these strings are equal. So also hashlib should confirm this:

Continuous integration with GitLab and Docker

Tue, 11 Oct 2016 00:00:00 +0000

I’ve given a talk about continuous integration with GitLab and Docker on Developer Meetup VlbgWebDev. The Meetup takes place every second Tuesday every Month and i’ve met some old colleagues there.

It was a nice Meetup with lots of discussions, that’s why i published my sample code to GitLab and my slides to Speakerdeck. First intention was to do some Pyramid demo but after I’ve heard there are some Flask people, I’ve added Flask demos as well :-)

Weblate and i18next json files

Fri, 29 Apr 2016 00:00:00 +0000

There’s a bug report on github about JSON format breaks i18next files due to automatic restructuring. Actually it flattens your i18next JSON file and makes it unusable for further usage.

So after starting with a JSON file like:

{
"navbar": {
"reward": "Rewards",
"about": "About"
}
}

you end up with a committed file which was transformed into this:

{
"navbar.reward": "Rewards",
"navbar.about": "About"
}

Use Weblate PRE_COMMIT_SCRIPTS

Weblate offers script hooks before or after a repository action happens. I’m using one of those hooks to execute a short Python script which reformats JSON again to be usable in i18next translation tools again. The script below is also available on Github:

GitLab as OAuth provider for Weblate

Thu, 28 Apr 2016 00:00:00 +0000

Weblate is built on Django and you can use a lot of authentication plugins. After my contribution to django-allauth and the backporting work of glensc and omab to django-social-auth it’s very easy to integrate [GitLab]’s authentication provider with Weblate

GitLab

All you have to do on GitLab is configuration through webinterface. Go to https://gitlab.com/profile/applications and register a new application.

Name: Weblate
Redirect URI: `https://weblate.yourdomain.tld/accounts/complete/GitLab/``

After registering your new application you’ll get your Application Key and Secret. You’ll need this later for configuration

Using SVN in eclipse on Mac OS X

Mon, 03 Feb 2014 00:00:00 +0000

After upgrading subversion client my eclipse could not find javahl bindings any more, because i previously installed the CollabNet ones, which are currently available for subversion 1.6.x only. The bad thing about this is Subclipse does not work any more, until you’ll install these libraries using the following solutions.

There may be one for macports as well, but i’m using homebrew and nothing else any more :-)

Installing through homebrew is not as easy as installing through macports because they are not available as single package, but you can add a --java option to the homebrew command to build required javahlbinding together with subversion. This will show up the following warning, because you should use a universal build.

Image filesize validator for dexterity content types in Plone

Fri, 31 May 2013 00:00:00 +0000

When using Plone 4.3 with plone.app.contenttypes you might want to limit max filesize for images by implementing a custom z3c.form validator. In ATCT we were able to limit max image filesize by overwriting ATCT config file or setting in portal_atct tool.

Using webserver

The simplest way to achieve a max-upload size is to just limit client_max_body_size in nginx webserver (or simliar in apache).

But my current solution is really different. It creates a validator for a special fieldtype, in our special case a INamedBlobImageField.

Use Babel to translate your python package

Mon, 01 Apr 2013 00:00:00 +0000

I use egg-containing buildouts for all my Plone packages. Since i18ndude does not extract msgids from zcml files, i tried Babel and succeeded.

Problem

You want to translate titles in Plone’s diplay menu by adding browser:menuItem in your zcml file as shown here.

<configure
xmlns="http://namespaces.zope.org/zope"
xmlns:browser="http://namespaces.zope.org/browser"
xmlns:plone="http://namespaces.plone.org/plone"
xmlns:i18n="http://namespaces.zope.org/i18n"
i18n:domain="my.package">
<browser:page
for="plone.folder.interfaces.IOrderableFolder"
name="a_new_view"
class=".demo_view.DemoView"
permission="zope2.View"
template="templates/demo_view.pt"
/>
<!-- Entry in display menu -->
<browser:menuItem
for="plone.folder.interfaces.IOrderableFolder"
menu="plone_displayviews"
title="A new view"
action="@@a_new_view"
description="I want a view with translated title and description"
i18n:attributes="title; description"
/>
</configure>

i18ndude does not extract message ids from zcml files, so i tried a diffent solution using Babel.

Turbo twitter Bootstrap theming for Plone using Diazo

Tue, 11 Dec 2012 00:00:00 +0000

When building a new Plone theme based on Twitter Bootstrap without modifying core templates you have to do a lot of xsl-styling. Hopefully this article can help you getting started.

Building a diazo theme

You should have basic knowledge of how to build a diazo theme in Plone. So you will have a file called rules.xml and hopefully your Twitter Bootstrap javascript and css code already plugged into Plone resource registries. An installable Plone product is available here.

Plone test development with robotframework and RIDE

Thu, 29 Nov 2012 00:00:00 +0000

Running acceptance tests in Plone

Test driven development (TDD) can be a pain. If you are working with Zope and Plone most of time gets consumed starting up Zope running setUp method and rolling back transaction in tearDown method.

So why not use a running already set-up instance for test-development? Today I will show you how to reach that goal. It’s not as hard as it seems.

Use buildout

Of course we use buildout to get a custom script which we can run through RIDE. I configured a new pybot script, the basic idea has been implemented in plone.act¹ module already.

Howto use buildout version pins?

Wed, 14 Nov 2012 00:00:00 +0000

When using Plone with buildout you want to be able to reproduce your environment with the same set of eggs, even if there were some new releases in between.

buildout 2.x

For buildout 2.x they implemented a lot of improvements for pinning versions.

allow-picked-versions allow buildout to download pinned packages only. Not pinned but required packages will raise an Exception.
show-picked-versions shows a summary of picked versions after your first buildout run. You can copy'n'paste this into a versions.cfg file.

buildout 1.x

Manually

Without pinning your eggs it will be impossible to reproduce the deployed buildout configuration in future. Use a this buildout extension for future projects to avoid this pitfall:

Install Robot Framework RIDE through homebrew

Tue, 16 Oct 2012 00:00:00 +0000

Python, wxPython and robotframework RIDE on your macOS.

Update 2016, 23 Feb

Fixed wxPython installation so it works with most recent versions available today.

Update 2017, 03 Jul

Seems this installation method no longer works on >= MacOS Sierra. Please use a docker image and Ride in browser.

wxPython

Since wxPython is available on homebrew you don’t have to install the package provided on the wxPython website.

$ brew info wxpython
wxPython: stable 3.0.2.0 (bottled)
Python bindings for wxWidgets
https://www.wxwidgets.org/
/usr/local/Cellar/wxPython/3.0.2.0 (1,107 files, 38.2M)
Poured from bottle
From: https://github.com/Homebrew/homebrew/blob/master/Library/Formula/wxpython.rb
==> Dependencies
Required: wxmac ✔
==> Options
--universal
$ brew install wxpython

This will take some time so be patient.

Change minute_step in Archetypes DateTimeWidget

Tue, 31 Aug 2010 00:00:00 +0000

Sometimes you need minutes which could not be divided by 5 which Plone doesn’t select you by default. You don’t need to rewrite core components in Archetypes DateTimeWidget to change this.

Prerequisites

You need your own BrowserLayer (a skin or theme product) or a contenttype with at least one marker interface to customize on.

Howto

Create a file called datecomponents.py in your browser directory with following content:

from plone.app.form.widgets import datecomponents
class DateComponents(datecomponents.DateComponents):
def result(self, date=None, use_ampm=False, starting_year=None,
ending_year=None, future_years=None, minute_step=5):
# Change minute steps to 1
return super(DateComponents, self).result(date, use_ampm,
starting_year, ending_year,
future_years, 1)

… and register it for you browserlayer (replace IThemeSpecific by your layer interface) or for a type you want to customize its widget (replace for='*' with your types interface), thats it.

Howto send mails from localhost through a smarthost on macOS

Wed, 07 Apr 2010 00:00:00 +0000

You’re getting weird errors like “Connection refused” from your smtp library? By reconfiguring your local mailer to send mails through a smarthost you can solve this issue.

Setting up a smarthost

Setting up your macOS to use a smarthost to forward mails to is not very hard. I found a nice tutorial on fuerstnet.de which was the base of this blog post.

Copy the lines:

$ sudo postconf -e relayhost=your.isps.mailserver smtp_use_tls=yes smtp_sasl_auth_enable=yes smtp_sasl_password_maps=hash:/etc/postfix/smtp_auth tls_random_source=dev:/dev/urandom smtp_sasl_security_options=noanonymous
$ sudo sh -c 'echo "your.isps.mailserver username:password" >> /etc/postfix/smtp_auth'
$ sudo chown root:wheel /etc/postfix/smtp_auth
$ sudo chmod 600 /etc/postfix/smtp_auth
$ sudo postmap hash:/etc/postfix/smtp_auth

Problems

I noticed postfix is quitting after 60 seconds. So if you send yourself a test email using: echo "test" | mail -s "test" your@email the postfix daemon will quit after 60 seconds and you’ll get these errors when trying to send mails through localhost in (for example) Plone:

Tell svn client to use a proxy-server

Tue, 09 Mar 2010 00:00:00 +0000

If you have or your server has to use a proxy server to access the internet you may have problems with your svn client although you aready set your http_proxy environment variable(s).

The solution to tell svn to use a proxy server is editing the ~/.subversion/servers file in your home directory and add the following lines or uncomment them, if they are already written into it:

[global]
# http-proxy-exceptions = *.exception.com, www.internal-site.org
http-proxy-host = my.proxy.server
http-proxy-port = 8080
# http-proxy-username = defaultusername
# http-proxy-password = defaultpassword
# http-compression = no
# http-auth-types = basic;digest;negotiate
# No http-timeout, so just use the builtin default.
# No neon-debug-mask, so neon debugging is disabled.
# ssl-authority-files = /path/to/CAcert.pem;/path/to/CAcert2.pem

Howto use custom color for collective.captcha?

Thu, 23 Apr 2009 00:00:00 +0000

How to use your custom color with collective.captcha and Plone? This tutorial shows you an easy way to customize it using your own BrowserLayer.

collective.captcha uses a BrowserView to display it’s captcha image. Therefore it’s very easy to customize it for your own theme because you can subclass that BrowserView for your own theme layer which is a BrowserLayer. You can change colors and/or fonts as you need it for your custom theme without doing any code change on the original collective.captcha package.