There are only two hard things in Computer Science: cache invalidation and naming things.
— Phil Karlton

IT is my daily life. And this quote is so true. Lately, I’ve been thinking much more than usual about naming, and that names really matter. This led to some refactoring activities.

Why Is Naming Important?

When a person with German mother tongue hears the word “eagle”, she or he automatically associates it with a “hedgehog”. Simply because the German word for it (“Igel”) is pronounced exactly the same. Of course, language skills and the concrete context play a role. The point is, a wrong association is likely. When we give a name to a thing, we basically want to avoid such false associations. In the best case, they are not helpful. In the worst case, this leads to rejection, as the next example shows.

In 1982 Mitsubishi Motors launched a SUV with the name “Pajero”. This name had to be changed in some regions, because “pajero” means “wanker” in Spanish. This example also shows that it is more important what others think about a name than we do.

In IT we have to name many things. Databases, schemas, tables, columns, views, packages, triggers, variables, fields, methods, classes, modules, components, products, etc. etc. Using an established name with a known and accepted definition help others to understand it better.

When we use a name, it is actually associated with a definition and properties, whether we like it or not. When names have a common and widely accepted meaning, it simplifies the communication. For example “banana”. Everybody knows what it means. Merriam-Webster’s definition is:

An elongated usually tapering tropical fruit with soft pulpy flesh enclosed in a soft usually yellow rind.

And I am sure that each of us could add a few characteristics to this definition.

Why Is Naming Difficult?

A name must fulfil many characteristics. For example

• Short
• Fitting (naturally relates to the intended meaning, and characteristics)
• Easy to spell, pronounce, remember
• Not associated with unwanted characteristics
• Common and widely accepted meaning and definition, that fits the intention (for names without commercial value)
• New, not used already (for marketable names)

Depending on context there are some goal conflicts. However, even without a major conflict, it is difficult to name something adequately in the early stages. Because we do not know enough about the thing we want to name. Hence, we use an iterative approach. We name something (e.g. an entity, package or class) and while working on it we find out that the name does not fit (anymore) and we change it. Maybe we split the thing and have to name now two things, etc. etc.

Finding a fitting name means doing some research. How have others named that thing? What is the definition of it? Does it fit 100 percent? This is an interesting and instructive work. In any case, it takes time. And at the time we need a new name, we want it now (e.g. when a wizard asks for a name). We can always rename it later, right? – Technically yes. And often we do. But the longer we wait, the less likely we are renaming.

Are Some Names More Important Than Others?

Yes. The more visible a name is the more important it is.

For example, the names behind an API are very easy to change. We do not have to ask anyone before changing it. It’s no problem as long as the API provides the same results. That’s one of the reasons we strive for tight APIs, right? To get some leeway.

As soon as others are involved, we are not fully in control of the change anymore. For example, when I change a name in one of my blog posts, this change is visible immediately to everyone visiting my blog. But I cannot control the caches of others, like search engines, blog mirrors and other services that copy web content to third-party storages. Remember, cache invalidation is the other hard thing in IT.

As a consequence, before we release an artefact that becomes visible to others, we should take some time to verify the used names. We cannot take back what we’ve said (at least not completely). However, we are in control of what we say in the future.

Banned Names on This Blog

Some terms (names) were discussed recently (again) due to a series of sad events. I used these terms as well. I never really thought about them as “bad”. However, I’ve changed my mind. I’m part of the problem. And I do not like it. One thing I can do is to stop using terms, that a large group of people associate with slavery and racism. No big deal, right?

This is another quote I like very much:

One cannot not communicate
— Paul Watzlawick

It is difficult to draw a line for certain terms. However, I believe that “you cannot not decide”. You decide either explicitly or implicitly. Of course, very seldom something is purely black or white. It’s much more often a shade of grey. Some decision take some time. And that’s okay. But it is impossible to postpone a decision forever. At a certain point, it becomes a decision.

So, I decided to decommission some terms on this blog and introduce new ones. Here’s the list:

Finding alternative names was surprisingly easy because others had already done the work and defined alternative names. They existed for years…

Master Data

However, finding an alternative for master data was harder. I reached out to my friends on Twitter. And got some helpful feedback. Finally, Robert Marti suggested having a look at Malcolm Chisholm‘s book Managing Reference Data in Enterprise Databases. On page 258ff the different data classes are defined and explained. The book is from 2000. In the meantime Malcolm Chisholm has published revised definitions here and here.

In the next subchapter, I repeat the definition of the data groups defined by Malcolm Chisholm on slide 5 in this deck. I like these definitions and plan to use them in the future.

Metadata

The data that describes all aspects of an enterprise’s information assets, and enables the enterprise to effectively use and manage these assets.

Here it is confined to the structure of databases. Found in a database’s system catalog. Sometimes included in database tables.

Reference Data

Any kind of data that is used solely to categorize other data found in a database, or solely for relating data in a database to information beyond the boundaries of the enterprise.

Codes and descriptions. Tables containing this data usually have just a few rows and columns.

Transaction Structure Data

Data that represents the direct participants in a transaction, and which must be present before a transaction fires.

The parties to the transactions of the enterprise. E.g. Customer, Product.

Enterprise Structure Data

Data that permits business activity to be reported and/or analyzed by business responsibility.

Typically, data that describes the structure of the enterprise. E.g. organizational or financial structure.

Transaction Activity Data

Data that represents the operations an enterprise carries out.

Traditional focus of IT – in many enterprises the only focus.

Transaction Audit Data

Data that tracks the life cycle of individual transactions.

Includes application logs, database logs, web server logs.

Summary

You use a name to simplify communication. A name is a proxy for a longer definition and meaning. If the meaning is badly received by others and especially by the target community, this does not simplify communication. Using a different name sounds like a simple solution. Why not, if changing a name is simple enough?

In this case, I only had to edit a few blog posts. I handled them like typos. This means that I did not add any update information. I also had to register new URL redirects. That was straightforward. However, changing the branch name in 26 GitHub repositories was a bit more work than anticipated, because I also had to change URLs in several related files. For certain GitHub pages, I had to keep a non-default master branch. I suppose that sooner or later GitHub will allow me to get rid of them as well. If I had to change more repositories, I would probably automate this task.

Most of the time I spent finding an alternative name for “master data”. In the end, I learned something new and found good names and definitions. That will help me in the future.

The post Names Matter appeared first on Philipp Salvisberg's Blog.

Here is the table of contents of the major steps.

1. Install Docker
2. Create SonarQube and Jenkins Container
3. Install PL/SQL Cop for SonarQube
4. Install PL/SQL Cop (Command Line Utility)
5. Configure SonarQube
6. Configure Jenkins
7. Create Code Analysis Job
8. View the Result in SonarQube
9. Summary

In the summary of this post, you find an audio-less video completing the process within 3.5 minutes.

1. Install Docker

I’m assuming that you already have installed Docker on your machine. If not, download and install the free Docker Community Edition including Docker Compose. I’m going to use Docker for Mac. But you may use Docker for Windows or a Docker server for one of the various supported Linux distributions. It is also possible to use a cloud provider, but the installation procedure will differ slightly.

You are ready for the next steps when the command

docker run --rm hello-world

produces this output

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
3. The Docker daemon created a new container from that image which runs the
   executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent it
   to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://cloud.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/engine/userguide/

in your terminal window.

2. Create SonarQube and Jenkins Container

In this step, we create a container for SonarQube and another one for Jenkins. Jenkins will need to communicate with SonarQube. Docker Compose allows us to use hostnames instead of hard-coded IP addresses and to manage all involved containers together in a single YAML configuration file.

Create a “docker-compose.yml” file in a “plsqlcop” directory. The directory name is used to derive names for containers and volumes. In this case, the volumes will be named “plsqlcop_sonardata” and “plsqlcop_cidata”. For the container names, the default naming has been overridden.

version: "3.0"
services:
   sonar:
      image: "sonarqube:5.1.2"
      container_name: "sonar"
      environment:
         - SONARQUBE_JDBC_USERNAME=sonar
         - SONARQUBE_JDBC_PASSWORD=sonar
         - SONARQUBE_JDBC_URL=
      ports:
         - "9010:9000"
      volumes:
         - "sonardata:/opt/sonarqube/data"
   ci:
      image: "jenkins/jenkins:2.79"
      container_name: "ci"
      ports:
         - "9020:8080"
      volumes:
         - "cidata:/var/jenkins_home"
volumes:
   sonardata:
      external: false
   cidata:
      external: false

Make sure you are located in the directory where the “docker-compose.yml” file is stored. Then create the containers in the background by running

docker-compose up -d

You should get an output similar to the following:

Creating volume "plsqlcop_sonardata" with default driver
Creating volume "plsqlcop_cidata" with default driver
Creating ci ... 
Creating sonar ... 
Creating sonar
Creating sonar ... done

3. Install PL/SQL Cop for SonarQube

To install the current version of PL/SQL Cop for SonarQube within the “sonar” container run

docker exec sonar wget --no-check-certificate https://www.salvis.com/blog?ddownload=6822 -O \
/opt/sonarqube/extensions/plugins/sonar-plsql-cop-plugin-2.1.1.jar

Windows users please note, that the \ (backslash) at the end of the first line is just the Unix line-continuation character (as the ` (grave accent) in PowerShell), you should omit it. The wget command will be executed within the sonar container.

To load the plugin we need to restart the container.

docker restart sonar

4. Install PL/SQL Cop (Command Line Utility)

To install the current version of PL/SQL Cop within the “ci” container run

docker exec -u 0 ci wget --no-check-certificate https://www.salvis.com/blog/?ddownload=6680 -O /opt/tvdcc.zip
docker exec -u 0 ci unzip /opt/tvdcc.zip -d /opt 
docker exec -u 0 ci bash -c "mv /opt/tvdcc-* /opt/tvdcc"  
docker exec -u 0 ci rm /opt/tvdcc.zip

Why do we need to install this component in the “ci” container and not in the “sonar” container? Well, the SonarQube scanner is executed in the “ci” container and gets the code analyzer from the SonarQube server. This increases the scalability of the code analyzers since just the analysis reports need to be sent to the SonarQube server. However, the PL/SQL Cop SonarQube plugin is a wrapper to the command line utility. This means that every Jenkins agent needs to have PL/SQL Cop installed in the location configured on the SonarQube server.

In an environment with multiple servers and multiple Jenkins agents, the PL/SQL Cop installations need to be identical on every agent.

5. Configure SonarQube

Open “http://localhost:9010” in your web browser.

Press “Log in” in the upper right corner and login into SonarQube with the username “admin” and password “admin”.

Click on “Settings” and the Category “Trivadis PL/SQL Cop” and change the “Path to PL/SQL Cop command line tvdcc executable” to “/opt/tvdcc/tvdcc.sh”.

Press “Save Trivadis PL/SQL Cop Settings” and your done.

6. Configure Jenkins

Open “http://localhost:9020” in your web browser.

Jenkins generated during the container creation a password for the user admin. To get this password execute in a terminal window the following:

docker exec ci cat /var/jenkins_home/secrets/initialAdminPassword

Copy the output into the clipboard and paste it into the “Administrator password” field in the browser window and press “Continue”.

Press on “Install Suggested Plugins”

Wait until all plugins are installed.

Enter the requested fields and press “Save and Finish”. And on the next page Click on “Start Using Jenkins”

Click on “Mange Jenkins”.

Click on “Manage Plugins”.

Select “SonarQube Scanner for Jenkins” in the “Available” tab. Use the filter to find the entry. Press “Install without restart”.

Click on “Go back to top page” when the plugins have been installed successfully.

Click on “Manage Jenkins” (again) and then on “Global Tool Configuration”.

Within the Global Tool Configuration click on “Add SonarQube Scanner”, add a name for the scanner and press “Save”.

Click on “Configure System”.

Scroll down to the “SonarQube servers” section and click on “Add SonarQube”.

Enter “SonarQube 5.1.2” for name, enter “http://sonar:9000” for “Server URL”, select “5.1 or lower” in “Server version” and enter “jdbc:h2:tcp://sonar/sonar” for “Database URL”. Click on “Save” to complete the Jenkins configuration.

Please note, that we are accessing the host sonar via the internal network Docker Compose has set up for us. Therefore we have to use the default port 9000 and not port 9010 which we are using from outside, e.g. to access SonarQube in the web browser.

7. Create Code Analysis Job

Click on “create new jobs”.

Enter “plscope-utils” and click on “Freestyle project”

Click on the “Source Code Management” tab, select “Git”, enter “https://github.com/PhilippSalvisberg/plscope-utils” for “Repository URL”, add “Sparse Checkout paths” in “Additional Behaviours”, enter “database/utils” for “Paths” and click on “Apply”.

Click on “Build Triggers” tab, select “Poll SCM”. Enter “H/15 * * * *” for “Schedule” and press “Apply”. This will look for changes in the Git repository every 15 minutes and starts a build if a change is detected.

Click on “Build” tab and Scroll down to the Build section and add a “Execute SonarQube Scanner” build step.

Copy the following lines into the “Analysis properties”.

sonar.projectKey=plscope-utils:main
sonar.projectName=plscope-utils
sonar.projectVersion=0.5.0
sonar.sources=database
sonar.language=plsqlcop

and click on “Save”.

Wait until the initial build is started or click on “Build Now” and then on “#1” (the first build job in the Build History).

Click on “Console Output”.

The analysis result has been stored successfully in SonarQube and can now be queried via http://localhost:9010 .

Please note, that the link to SonarQube provided in the console output is not working, since it is referring a port of the internal Docker Compose network which is not accessible to your browser. However, it is possible to configure the external IP address and port for SonarQube in Jenkins and than this link will work.

8. View the Result in SonarQube

Open “http://localhost:9010” in your web browser.

Click on “plscope-utils”,  select the “Issues” tab for this project and select all rules.

Click on “>” at the right side of an issue to see the source code line causing this issue.

See https://www.sonarqube.org/features/clean-code/ for more information.

9. Summary

Setting up a continuous code quality inspection environment for a PL/SQL project with Docker is quite simple. The audio-less video documents the complete process.

Granted, for production use, you need to use a different database as SonarQube backbone, define some roles, manage users and probably integrate an Active Directory. That should not be too difficult, at least not technically.

But the most challenging part will be to agree on some rules, adapt your development process and improve your code quality over time. So, let’s get started.

The post Continuous Code Quality for PL/SQL with Docker appeared first on Philipp Salvisberg's Blog.