Code Analysis Archives - Philipp Salvisberg's Blog https://www.salvis.com/blog/tag/code-analysis/ Database-centric development Wed, 29 Jul 2026 04:07:53 +0000 en-US hourly 1 https://wordpress.org/?v=7.0.2 https://www.salvis.com/blog/wp-content/uploads/2014/04/favicon.png Code Analysis Archives - Philipp Salvisberg's Blog https://www.salvis.com/blog/tag/code-analysis/ 32 32 Detecting Security Vulnerabilities With the APEXlang Parser https://www.salvis.com/blog/2026/06/29/detecting-security-vulnerabilities-with-the-apexlang-parser/ https://www.salvis.com/blog/2026/06/29/detecting-security-vulnerabilities-with-the-apexlang-parser/#respond Mon, 29 Jun 2026 12:12:21 +0000 https://www.salvis.com/blog/?p=19914 Introduction A parser for the APEXlang grammar enables the development of tools beyond the scope of the APEXlang compiler included in SQLcl, SQL Developer for VS Code, and ORDS. These tools can perform static code analysis, convert code, generate documentation, and visualise different aspects of an APEX application. A linter, for example, […]

The post Detecting Security Vulnerabilities With the APEXlang Parser appeared first on Philipp Salvisberg's Blog.

]]> Introduction

A parser for the APEXlang grammar enables the development of tools beyond the scope of the APEXlang compiler included in SQLcl, SQL Developer for VS Code, and ORDS. These tools can perform static code analysis, convert code, generate documentation, and visualise different aspects of an APEX application.

A linter, for example, can check whether an APEXlang file conforms to defined quality standards. These standards may cover project- or company-specific conventions as well as more general concerns, such as detecting potential security vulnerabilities.

In this blog post, I explain the elements of an APEXlang file that are the basis of Grisselbav’s APEXlang parser. Then I demonstrate how to use the parser in a simple Java program to detect a security vulnerability.

Elements of an APEXlang File

An APEXlang file consists of three basic building blocks: components, properties, and groups. These elements can also be nested, as shown later.

Here’s an excerpt of Oracle’s Universal Theme demo application.apx file:

1) Excerpt of UT’s application.apx
app UT (
    name: Universal Theme 26.1 Reference
    version: 26.1.0
    group: @universal-theme
    logo {
        type: text
        text: Universal Theme
    }
    // ...removed code...
)
Component

There is one component in this example. It starts on line 1 and ends on line 10. app is the type of the component and UT the component name.

A component body starts with a left parenthesis ( and ends with a right parenthesis ).

Property

In this example, we have 5 properties. The property keys are name, version, group, type and text. The value of a property follows a colon :. For name The value is Universal Theme 26.1 Reference, for version the value is 26.1.0 and so on.

Please note that a property starts on a new line, and the property value starts after the colon and ends on a new line. This way, no delimiter characters are required for most property values. However, if leading or trailing spaces are significant, you have to pass the property value as a single-line string, which is enclosed in double quotes.

Group (of properties)

The properties type and text are part of a group named logo. The group in this example covers lines 5 to 8.

A group body starts with an open curly bracket { and ends with a close curly bracket }.

Properties which are not part of a group are called direct properties. This means they are defined directly in a component. Examples of direct properties are name, version and group.

Nested Elements

We have seen that a component may contain properties and groups. But it is also possible to nest elements.

  • Components may contain other components besides properties and groups
  • Property values may contain groups besides simple values

There is no limit to the number of levels for nested elements.

Here’s an example:

2) Nested Component and Group
app UT (
    // ...removed code...
    pwaShortcut getting-started (
        name: Getting Started
        sequence: 10
        shortcut {
            target: {
                page: 500
            }
            description: Getting Started Page - Initial Page
        }
        comments {
            comments: -
        }
    )
   // ...removed code...
)

A nested component starts on line 3 and a nested group on line 7.

APEXlang Grammar

apxFile rule of APEXlang grammar

While the grammar documented in apexlang. ebnf of the API Reference defines both the language structure and the valid APEXlang elements, the grammar used by Grisselbav’s APEXlang parser is limited to structural concerns.

Because the parser focuses solely on syntax rather than semantic validation, the grammar remains remarkably compact.

Click on the image above to view all the syntax diagrams. Alternatively, view the ANTLR4 source files, ApexLangLexer.g4 and ApexLangParser.g4, which were used to generate the parser available on Maven Central.

APEX-SERT

APEX-SERT is an APEX application that scans a selected APEX application for security vulnerabilities. A vulnerability is detected by querying APEX dictionary views. All rules are defined in the APEX-SERT Rules.json file.

To demonstrate a practical use case, let’s look at the security check “Embed in Frames” implemented by APEX-SERT.

When the property “Embed in Frames” is set to Allow, the application may be vulnerable to “clickjacking” attacks as explained in the help text of the page designer. See screenshot below.

I changed the original value from Allow from same origin to Allow to simulate a security vulnerability.

APEX-SERT runs a query similar to the following to find violations of this rule:

3) Find Violations with SQL
select application_id, application_name, browser_frame 
  from apex_applications
 where browser_frame not in ('Deny', 'Allow from same origin');
APPLICATION_ID APPLICATION_NAME               BROWSER_FRAME
-------------- ------------------------------ -------------
        101252 Universal Theme 26.1 Reference Allow

So, the value of Embed in Frames is provided in the browser_frame column of the APEX dictionary view apex_applications.

But where can we find this information in the APEXlang files?

Link Page Designer Property to APEXlang

We find the “Embed in Frames” property in the page designer by navigating to “Shared Components” -> “Application Definition” -> “Security” -> “Browser Security”.

According to the apexlang. ebnf there is a security group within the app component. Here’s the relevant excerpt:

3) Property embedInFrames within apexlang.ebnf
<app-security> ::= <indent> "security" <ws> "{" <line-end> { <app-security-property-line> } <indent> "}" <line-end>
<app-security-property-line> ::= <indent> <app-security-property> <line-end>
<app-security-property> ::= "deepLinking" ":" <ws> ( "true" | "false" ) (* required; type: SELECT LIST *)
  | "enableDictation" ":" <ws> <boolean> (* required; type: YES NO *)
  | "browserCache" ":" <ws> ( "true" | "false" ) (* required; type: SELECT LIST *)
  | "embedInFrames" ":" <ws> ( "deny" | "allowSameOrigin" | "allow" ) (* required; type: SELECT LIST *)
  | "referrerPolicy" ":" <ws> ( "noReferrer" | "noReferrerWhenDowngrade" | "origin" | "originWhenCrossOrigin" | "sameOrigin" | "strictOrigin" | "strictOriginWhenCrossOrigin" | "unsafeUrl" ) (* required; type: SELECT LIST *)
  | "htmlEscapingMode" ":" <ws> ( "basic" | "extended" ) (* required; type: SELECT LIST *)
  | "httpResponseHeaders" ":" <ws> <multiline-string> (* type: TEXT EDITOR *)
  | "runtimeApiUsage" ":" <ws> <array-of-string-like-value> (* type: CHECKBOXES *)

We find on line 352 the definition of the property embedInFrames with a list of valid values. In this case, the property key is unique. This means that no other component or group uses the property key embedInFrames. Therefore, we can search the folder containing the .apx files for the string embedInFrames: to find the position of this property.

The matching property is located in the application.apx file.

Here’s an excerpt.

4) Property embedInFrames in application.apx
    authorization {
        runOnPublicPages: true
    }
    security {
        deepLinking: true
        embedInFrames: allow
        runtimeApiUsage: modifyThisApp
    }
    sessionStateProtection {
        allowUrlsCreatedAfter: 1999-08-04T00:00:00
        checksumSalt: 75BAAC4002F8CA56EF54FD242CCE7719B1AB85BE339E930260B3EC8EA3879365
    }

Please note that properties with default values are skipped when an APEX application is exported. Therefore, we would not see the embedInFrames property if its value were Deny.

Find Vulnerability With APEXlang Parser

We now have all the information we need. We can write a small demo program. This program will find this vulnerability in an APEXlang file.

The parser exposes the parse tree through ANTLR4-generated context classes. Each parser rule becomes a context class. For example, the class ApexLangParser.PropertyContext represents the property parser rule. This makes it easy to traverse the ApexLangDocument using Java streams.

5) DemoFindVulnerability.java
//DEPS com.grisselbav:apexlang-parser:0.2.0

import com.grisselbav.apexlang.grammar.*;

class DemoFindVulnerability {
    public static void main(String[] args) {
        var apxSource = """
                app UT (
                    name: Universal Theme 26.1 Reference
                    version: 26.1.0
                    // ... removed code ...
                    authorization {
                        runOnPublicPages: true
                    }
                    security {
                        deepLinking: true
                        embedInFrames: allow
                        runtimeApiUsage: modifyThisApp
                    }
                    sessionStateProtection {
                        allowUrlsCreatedAfter: 1999-08-04T00:00:00
                        checksumSalt: 75BAAC4002F8CA56EF54FD242CCE7719B1AB85BE339E930260B3EC8EA3879365
                    }
                    // ... removed code ...
                )
                """;
        var doc = ApexLangDocument.parse(apxSource);
        var violations = doc.getAllContentsOfType(ApexLangParser.PropertyContext.class)
                .stream()
                .filter(p -> p.name.getText().equals("embedInFrames")
                        && p.value().getText().equals("allow"))
                .toList();
        for (var violation : violations) {
            System.err.println("Found vulnerability: " + violation.getText());
        }
    }
}

This program embeds the APEXlang source code to scan in the apxSource variable. To run the program, save it as DemoFindVulnerability.java, then run it with JBang. This will automatically resolve the Maven dependency on the first line.

jbang DemoFindVulnerability.java
[jbang] Resolving dependencies...
[jbang]    com.grisselbav:apexlang-parser:0.2.0
[jbang] Dependencies resolved
[jbang] Building jar for DemoFindVulnerability.java...
Found vulnerability: 
        embedInFrames: allow

Are Regular Expressions An Alternative?

In this case, we could have used a regular expression to identify the violation.

However, as soon as the rules become more complicated, e.g. if we want to ensure that the property is part of the security group and the security group is part of the app component, regular expressions are no longer suited. But a parser is. Because we can navigate the parse tree and adjust the filter settings as follows.

6) DemoFindVulnerability2.java with extended filter
//DEPS com.grisselbav:apexlang-parser:0.2.0

import com.grisselbav.apexlang.grammar.*;

class DemoFindVulnerability2 {
    public static void main(String[] args) {
        var apxSource = """
                app UT (
                    name: Universal Theme 26.1 Reference
                    version: 26.1.0
                    // ... removed code ...
                    authorization {
                        runOnPublicPages: true
                    }
                    security {
                        deepLinking: true
                        embedInFrames: allow
                        runtimeApiUsage: modifyThisApp
                    }
                    sessionStateProtection {
                        allowUrlsCreatedAfter: 1999-08-04T00:00:00
                        checksumSalt: 75BAAC4002F8CA56EF54FD242CCE7719B1AB85BE339E930260B3EC8EA3879365
                    }
                    // ... removed code ...
                )
                """;
        var doc = ApexLangDocument.parse(apxSource);
        var violations = doc.getAllContentsOfType(ApexLangParser.PropertyContext.class)
                .stream()
                .filter(p -> p.name.getText().equals("embedInFrames")
                        && p.value().getText().equals("allow")
                        && p.parent.parent instanceof ApexLangParser.GroupContext g
                        && g.name.getText().equals("security")
                        && g.parent.parent.parent instanceof ApexLangParser.ComponentContext c
                        && c.type.getText().equals("app"))
                .toList();
        for (var violation : violations) {
            System.err.println("Found vulnerability: " + violation.getText());
        }
    }
}

Integration Into dbLinter

The APEX-SERT “Embed in Frames” rule has been added to the dbLinter repository as rule A-0770: Never allow application pages within an HTML frame. The VS Code extension, the CLI and the SonarQube plugin now support checks implemented for APEXlang.

Here’s a short silent video that demonstrates how this security vulnerability is detected and quickly fixed.

Conclusion

The APEXlang grammar is intentionally simple through focusing on structural elements, leaving semantic validation, such as valid property values, to the APEXlang compiler in SQLcl, SQL Developer and VS Code.

Integrating the APEXlang parser into dbLinter was straightforward, as was implementing the first APEXlang-based dbLinter rule.

The next step is to identify which additional rules would be beneficial to bring to dbLinter. If you have any suggestions, please let me know. Even better, open a GitHub issue in the dbLinter GitHub repository.

Thank you.

Updated on 2026-07-29, fixed link to rule “Never allow application pages within an HTML frame” (A-1010 → A-0770).

The post Detecting Security Vulnerabilities With the APEXlang Parser appeared first on Philipp Salvisberg's Blog.

]]> https://www.salvis.com/blog/2026/06/29/detecting-security-vulnerabilities-with-the-apexlang-parser/feed/ 0 Fighting Bad PL/SQL & SQL with VS Code https://www.salvis.com/blog/2026/01/02/fighting-bad-pl-sql-sql-with-vs-code/ Fri, 02 Jan 2026 21:05:10 +0000 https://www.salvis.com/blog/?p=14289 The success of a project or product depends largely on the quality of the code. But how can I improve security, maintainability or performance? More importantly, how can I prevent code with quality defects from entering production? In addition to testing, code analysis plays a central role. These days, code can be […]

The post Fighting Bad PL/SQL & SQL with VS Code appeared first on Philipp Salvisberg's Blog.

]]> The success of a project or product depends largely on the quality of the code. But how can I improve security, maintainability or performance? More importantly, how can I prevent code with quality defects from entering production? In addition to testing, code analysis plays a central role. These days, code can be checked not only in CI/CD pipelines, but also during development. Modern IDEs, such as Visual Studio Code, can automatically detect quality issues and offer quick fixes. The goal is a fast feedback loop.

This blog post is based on my German article with the same title, published in issue 1/2026 of the Red Stack Magazin.

Measuring Software Quality

Imagine you are part of a team where code changes to a feature or bug fix branch are only merged into the develop or main branch following a review. Sometimes you are the author and sometimes one of the reviewers. I am currently part of a team that works in this way. In addition to quality assurance, knowledge sharing is also an important aspect. But how can we assess code quality?

Figure 1: WTFs/m

This well-known meme by Thom Holwerda says, “The only valid measurement of code quality is WTFs per minute.” This seems to be a good metric. However, this metric does have some flaws.

  • It is subjective.
  • It does not provide any information about what is being assessed.
  • It does not help new team members to adapt quickly.
  • And it cannot be automated in a meaningful way.

Sooner or later, a team has no choice but to agree on a set of rules. They can be informal or formal. Typical coding guidelines cover topics such as naming conventions, formatting, code complexity, and language usage rules. The aim is to define and comply with quality standards relating to maintainability, reliability, performance and security. Written coding guidelines also simplify the onboarding of new team members.

Automation

Manually checking compliance with coding guidelines as part of a review is so time-consuming that no one can afford to do it properly. Therefore, we have to automate as many checks as possible. This is possible for many aspects. The rest still have to be checked manually.

In other words, we need a tool suite for automatic rule checking. dbLinter is such a tool suite. It is a new product developed and continuously improved by Grisselbav and United Codes. dbLinter uses the IslandSQL parser, which is updated quarterly with every release update of the Oracle AI Database for SQL, PL/SQL, SQL*Plus and SQLcl.

dbLinter is the successor to the decommissioned db* CODECOP suite. dbLinter consists of a central repository, a web GUI and client tools. These include a CLI, a SonarQube plugin and a VS Code extension.

Visual Studio Code Extension

The VS Code extension is compatible with all IDEs that implement the Visual Studio Code Extension API. Specifically: Visual Studio Code, Theia IDE, VSCodium, Cursor, Windsurf, Void, Kiro and Antigravity.

dbLinter for VS Code is available in both the Visual Studio Marketplace and the Open VSX Registry. Therefore, it can be conveniently installed directly from the IDE. The extension is free to use as part of the Anonymous and Starter plans (see dbLinter Pricing Plans).

It is important to note that your code is always analysed locally, meaning that it never leaves your network. The configuration is read from the central repository, enabling the client to identify the active rules and parameters. These configurations, along with others, are managed using the Web GUI.

Figure 2: dbLinter rule configuration in the Web GUI

dbLinter currently offers 187 rules. This includes all the rules from the decommissioned Trivadis PL/SQL & SQL Coding Guidelines. Figure 2 shows the configuration of my 14 favourite rules. A violation of one of these rules is a bug. There is almost no room for interpretation here. The following chapters provide two examples.

Top-N Queries with ROWNUM

The query in Listing 1 is based on the well-known EMP table owned by the database user SCOTT. It returns two rows.

Listing 1: Defective pre-12c-style top-n query
select ename,
       sal,
       hiredate,
       rownum as rank
  from emp
 where rownum <= 2
 order by sal desc;
ENAME             SAL HIREDATE         RANK
---------- ---------- ---------- ----------
ALLEN            1600 20.02.1981          2
SMITH             800 17.12.1980          1

However, the result barely meets the requirements. I would have expected the two employees with the highest salaries to be returned. Why is this not the case? The query is formulated in such a way that two rows are randomly selected and then sorted in descending order by salary. The result depends on the execution plan and the physical storage of the data.

Figure 3: Violation of rule G-3185 in VS Code

In VS Code, rule violations are displayed under Problems (see Figure 3). There is also a Core G-3185 link that opens the rule details in the Web GUI.

Figure 4: Rule G-3185 in the dbLinter Web GUI

Rule G-3185 in the web GUI is explained based on a similar case in the HR schema (see Figure 4). There are two solutions here. One that can be applied in all Oracle Database versions and one that works from version 12c onwards. Listing 2 shows the traditional solution, which uses an inline view. Of course, nowadays I would always prefer a solution using the row_limiting_clause.

We can also see that the rule is used in six configurations. Clicking on “(details)” opens a pop-up window in which changes can be made. This allows you to configure the rules directly from the IDE.

Listing 2: Fixed pre-12c-style top-n query
select e.*,
       rownum as rank
  from (
          select ename,
                 sal,
                 hiredate
            from emp
           order by sal desc
       ) e
 where rownum <= 2;
ENAME             SAL HIREDATE         RANK
---------- ---------- ---------- ----------
KING             5000 17.11.1981          1
SCOTT            3000 19.04.1987          2

NVL vs. COALESCE

One often underestimated rule is “G-4230: Always use a COALESCE instead of a NVL command, if parameter 2 of the NVL function is a function call or a SELECT statement”. This rule is about performance, which is relevant to all of us, especially when it is poor. This is why this rule is important.

Listing 3: Bad performance with NVL
with
   function slow_function return integer is
   begin
      sys.dbms_session.sleep(0.2);
      return null;
   end slow_function;
select ename, nvl(mgr, slow_function()) as mgr
  from emp;

Listing 3 demonstrates a case in which the use of NVL results in an execution time of approximately 2.8 seconds. If COALESCE is used instead of NVL, the execution time is only about 0.2 seconds. The reason is that NVL always evaluates the second parameter, even if the first parameter is not null. COALESCE works differently: the second parameter is only evaluated if the first parameter is null. This performance difference originates from the fact that the EMP table contains 14 rows, but only one of these returns an empty MGR.

In this case, using COALESCE is definitely the better solution. So, should NVL always be replaced by COALESCE? No. There are two special cases:

  1. NVL supports implicit data type conversion.
    select nvl(mgr, '1') from emp works, but select coalesce(mgr, '1') from emp causes an ORA-00932 error. However, implicit type conversions should be avoided anyway. The code should be adjusted accordingly.
  2. Optimisations of NVL can lead to better execution plans.
    Such optimisations currently only exist for NVL and not for COALESCE. Therefore, it is better to continue using NVL in these cases. See The Magical NVL Function by Monika Lewandowska for an example.
Quick Fixes

dbLinter knows these special cases. It offers appropriate alternative solutions. See Figure 5.

Figure 5: Quick fixes for G-4230 violation in VS Code.

dbLinter can apply quick fixes to different sections. For example, for a single violation of a rule, for all violations of a rule, or for all violations of all rules in the editor window.

The “@dbLinter ignore markers” are special comments used to ignore rule violations. There are various legitimate reasons for doing this. For example, in the case of false positives (dbLinter bugs) or exceptions to the rule. A comment such as -- @dbLinter ignore(G-4230): NVL forever can have the following scopes: line, statement or file, depending on where the comment is positioned.

Code Analysis and AI

The solution proposals in Figure 5, indicated by the yellow lamp symbol, are quick fixes from dbLinter. However, the last two menu options come from GitHub Copilot.

When Copilot suggests a solution to a rule violation, dbLinter provides the necessary context, including the file and the exact location of the violation, as well as the name of the violated rule. This gives Copilot access to the rule definition online, enabling it to suggest a reasonable solution in many cases. In other words, dbLinter enhances the support offered by tools such as Copilot, Cline and Windsurf.

Rules are useful for AI tools. They provide a framework that leads to better results. However, there is no guarantee that all rules will be followed. Ultimately, code generated or expanded by AI becomes part of our code base. This is why it is important to check AI-generated code according to the same rules as manually created code.

In other words, AI does not enforce standards. But dbLinter can do that.

Conclusion

Software quality is a matter of definition. dbLinter offers many rules. Not all of them are useful and applicable to every project. The dbLinter rules can therefore be seen as a large buffet. You choose what you like and what is healthy. But not too much at once. Nevertheless, there are some rules that, in my opinion, should be followed in every project. I have presented two of them in this blog post.

Install dbLinter for VS Code to identify the rules that will be most helpful for your projects. The rules in Figure 2 are certainly a good starting point.

If you have any questions, please feel free to contact me. Alternatively, you can create issues directly in the dbLinter GitHub repository.

The post Fighting Bad PL/SQL & SQL with VS Code appeared first on Philipp Salvisberg's Blog.

]]> Outer Joining With ANSI SQL-89 and SQL-92 https://www.salvis.com/blog/2025/07/12/outer-joining-with-ansi-sql-89-and-sql-92/ Sat, 12 Jul 2025 15:20:05 +0000 https://www.salvis.com/blog/?p=13988 Introduction I recently implemented a quick fix for the rule G-3130 to transform queries to ANSI SQL-92 syntax. One of the challenges was to handle Oracle’s outer-join syntax (+). In ANSI SQL-89, join and filter criteria are part of the WHERE clause. It’s not easy to distinguish them. In this blog post, […]

The post Outer Joining With ANSI SQL-89 and SQL-92 appeared first on Philipp Salvisberg's Blog.

]]> Introduction

I recently implemented a quick fix for the rule G-3130 to transform queries to ANSI SQL-92 syntax. One of the challenges was to handle Oracle’s outer-join syntax (+). In ANSI SQL-89, join and filter criteria are part of the WHERE clause. It’s not easy to distinguish them. In this blog post, I explain why it is important to do that when using outer joins, regardless of which join syntax you prefer.

Data Setup

All examples in this blog post are based on the tables DEPT and EMP, that were originally present in every Oracle Database instance as part of the SCOTT schema. Here’s a script to create these tables. The script requires an Oracle Database 23ai.

1) Create table DEPT and EMP
create table if not exists dept as select * from (values
          (10, 'ACCOUNTING', 'NEW YORK'),
          (20, 'RESEARCH',   'DALLAS'),
          (30, 'SALES',      'CHICAGO'),
          (40, 'OPERATIONS', 'BOSTON')
       ) s (deptno, dname, loc);

create table if not exists emp as select * from (values
          (7839, 'KING',   'PRESIDENT', null, date '1981-11-17', 5000, null, 10),
          (7566, 'JONES',  'MANAGER',   7839, date '1981-04-02', 2975, null, 20),
          (7698, 'BLAKE',  'MANAGER',   7839, date '1981-05-01', 2850, null, 30),
          (7782, 'CLARK',  'MANAGER',   7839, date '1981-06-09', 2450, null, 10),
          (7788, 'SCOTT',  'ANALYST',   7566, date '1987-04-19', 3000, null, 20),
          (7902, 'FORD',   'ANALYST',   7566, date '1981-12-03', 3000, null, 20),
          (7499, 'ALLEN',  'SALESMAN',  7698, date '1981-02-20', 1600,  300, 30),
          (7521, 'WARD',   'SALESMAN',  7698, date '1981-02-22', 1250,  500, 30),
          (7654, 'MARTIN', 'SALESMAN',  7698, date '1981-09-28', 1250, 1400, 30),
          (7844, 'TURNER', 'SALESMAN',  7698, date '1981-09-08', 1500,    0, 30),
          (7900, 'JAMES',  'CLERK',     7698, date '1981-12-03',  950, null, 30),
          (7934, 'MILLER', 'CLERK',     7782, date '1982-01-23', 1300, null, 10),
          (7369, 'SMITH',  'CLERK',     7902, date '1980-12-17',  800, null, 20),
          (7876, 'ADAMS',  'CLERK',     7788, date '1987-05-23', 1100, null, 20)
       ) s (empno, ename, job, mgr, hiredate, sal, comm, deptno);

Joins

The predecessor of the ANSI SQL-92 standard is the ANSI SQL-89 standard. Based on that standard, you could write joins only like this:

2) ANSI SQL-89 Join
select dept.deptno, emp.ename
  from dept, emp
 where emp.deptno = dept.deptno
 order by dept.deptno, emp.ename;
    DEPTNO ENAME 
---------- ------
        10 CLARK 
        10 KING  
        10 MILLER
        20 ADAMS 
        20 FORD  
        20 JONES 
        20 SCOTT 
        20 SMITH 
        30 ALLEN 
        30 BLAKE 
        30 JAMES 
        30 MARTIN
        30 TURNER
        30 WARD  

14 rows selected.

In the FROM clause, we list the tables to be joined, and in the WHERE clause, we define the JOIN criteria. There are no filter criteria in this case.

There are several ways to transform this query to ANSI SQL-92. One rather silly way, but very similar to the original ANSI SQL-89 syntax, is the use of an explicit CROSS JOIN.

3) ANSI SQL-92 Cross Join 
select dept.deptno, emp.ename
  from dept cross join emp
 where emp.deptno = dept.deptno
 order by dept.deptno, emp.ename;

While this is technically an ANSI SQL-92 compliant query, it’s not a good option. Why? – ANSI SQL-92 allows us to define the join criteria for each join. Using a CROSS JOIN implies that there is no join criterion between these tables, which is wrong.

A better option is to use an INNER JOIN, like in the next example.

4) ANSI SQL-92 Inner Join
select dept.deptno, emp.ename
  from dept join emp on emp.deptno = dept.deptno
 order by dept.deptno, emp.ename;

We see that the WHERE clause is gone. Why? – Because we do not have filter criteria in this query. No WHERE clause, no filter. As simple as that.

Outer Joins

Now let’s write an outer join with ANSI SQL-89. As I mentioned before, the standard back then only defined INNER JOINs. So, we need to work around it. One way is something like this:

5) ANSI SQL-89 Outer Join
select dept.deptno, emp.ename
  from dept, emp
 where emp.deptno = dept.deptno
union all
select dept.deptno, null
  from dept
 where not exists (
          select 1
            from emp
           where emp.deptno = dept.deptno
       )
 order by deptno, ename;
    DEPTNO ENAME 
---------- ------
        10 CLARK 
        10 KING  
        10 MILLER
        20 ADAMS 
        20 FORD  
        20 JONES 
        20 SCOTT 
        20 SMITH 
        30 ALLEN 
        30 BLAKE 
        30 JAMES 
        30 MARTIN
        30 TURNER
        30 WARD  
        40       

15 rows selected.

Phew, that’s a lot of work. It’s no surprise that database vendors came up with proprietary solutions for outer joins. For example:

  • *= and =* operators in Sybase/SQL Server
  • (+) operator in Db2 and the Oracle Database

Let’s use the Oracle join syntax.

6) Oracle-style Outer Join
select dept.deptno, emp.ename
  from dept, emp
 where emp.deptno (+) = dept.deptno
 order by dept.deptno, emp.ename;

Much simpler. EMP is outer joined due to the (+) operator.

And here is an ANSI SQL-92 compliant variant:

7) ANSI SQL-92 Outer Join
select dept.deptno, emp.ename
  from dept left join emp on emp.deptno = dept.deptno
 order by dept.deptno, emp.ename;

No WHERE clause, no filter. So far, so good.

Outer Joins With Additional Filters

The problem starts as soon as you use outer join queries with filter predicates. Here’s an example.

8a) Outer Join With Filter – Oracle-style
select dept.deptno, emp.ename, emp.hiredate
  from dept, emp
 where emp.deptno (+) = dept.deptno
   and emp.hiredate (+) > date '1981-12-03'
 order by deptno, ename;
    DEPTNO ENAME  HIREDATE  
---------- ------ ----------
        10 MILLER 23.01.1982
        20 ADAMS  23.05.1987
        20 SCOTT  19.04.1987
        30                  
        40
8b) Outer Join With Filter – ANSI SQL-92
select dept.deptno, emp.ename, emp.hiredate
  from dept
  left join emp on emp.deptno = dept.deptno
   and emp.hiredate > date '1981-12-03'
 order by deptno, ename;
    DEPTNO ENAME  HIREDATE  
---------- ------ ----------
        10 MILLER 23.01.1982
        20 ADAMS  23.05.1987
        20 SCOTT  19.04.1987
        30                  
        40

Both query variants use the predicate emp.hiredate > date '1981-12-03' as part of the outer join criteria. As a result, the DEPT rows 30 and 40 are shown with empty EMP columns.

The result is correct. But is this the result we want?

I don’t think so. One option is that we don’t want an outer join at all. Something like this:

9a) Inner Join With Filter – ANSI SQL-89
select dept.deptno, emp.ename, emp.hiredate
  from dept, emp
 where emp.deptno = dept.deptno
   and emp.hiredate > date '1981-12-03'
 order by deptno, ename;
    DEPTNO ENAME  HIREDATE  
---------- ------ ----------
        10 MILLER 23.01.1982
        20 ADAMS  23.05.1987
        20 SCOTT  19.04.1987
9b) Inner Join With Filter – ANSI SQL-92
select dept.deptno, emp.ename, emp.hiredate
  from dept
  join emp on emp.deptno = dept.deptno
 where emp.hiredate > date '1981-12-03'
 order by deptno, ename;
    DEPTNO ENAME  HIREDATE  
---------- ------ ----------
        10 MILLER 23.01.1982
        20 ADAMS  23.05.1987
        20 SCOTT  19.04.1987

Now we use the predicate emp.hiredate > date '1981-12-03' as a filter criterion. It’s applied after the join. The result would be the same if we used a LEFT JOIN in 9b).

In most cases, join criteria are based only on the foreign key-primary key relationships. If I see something else, I typically need to dig deeper to understand what the requirements of the query are.

So, maybe we want to see only the employees that match the filter, including departments that have no employees. For that, we could rewrite the query as follows:

10a) Outer Join With Filter – Oracle-style
select dept.deptno, emp.ename, emp.hiredate
  from dept, emp
 where emp.deptno (+) = dept.deptno
   and (emp.hiredate > date '1981-12-03' or emp.deptno is null)
 order by deptno, ename;
    DEPTNO ENAME  HIREDATE  
---------- ------ ----------
        10 MILLER 23.01.1982
        20 ADAMS  23.05.1987
        20 SCOTT  19.04.1987
        40
10b) Outer Join With Filter – ANSI SQL-92
select dept.deptno, emp.ename, emp.hiredate
  from dept left join emp
    on emp.deptno = dept.deptno
 where (emp.hiredate > date '1981-12-03' or emp.deptno is null)
 order by deptno, ename;
    DEPTNO ENAME  HIREDATE  
---------- ------ ----------
        10 MILLER 23.01.1982
        20 ADAMS  23.05.1987
        20 SCOTT  19.04.1987
        40

This example shows why it’s important to distinguish between join criteria and filter criteria. The ANSI SQL-92 syntax makes this easier. Unfortunately, there are still a few bugs and limitations when using ANSI-92 syntax in every Oracle Database version. But there are fewer and fewer of these problems with each new version.

Therefore, I use the ANSI SQL-92 join syntax whenever possible and try to document the reason when I cannot.

Use dbLinter to Change Queries to ANSI SQL-92 Join Syntax

dbLinter can find queries that do not use ANSI SQL-92 join syntax. Simply put, it searches for multiple items in the FROM clause. It also provides a quick fix to convert ANSI SQL-89 and Oracle-style joins to ANSI SQL-92 syntax. Here’s a screenshot with the previously discussed queries:

dbLinter for VS Code showing G-3130 violations and the pop-up window to apply quick fixes

After applying the quick fix to all G-3130 violations in the file, the result looks like this:

dbLinter for VS Code after applying the quick fix for all G-3130 violations

The result looks quite good, except that the predicate in the 8a) example is not moved to the join condition (see red arrow in the screenshot above). Why is that the case? – The quick fix only moves equality predicates and predicates that are not part of a Boolean OR condition to the join condition. If there are leftover outer join operators (+) in the WHERE clause, they are surrounded by a TODO comment.

To get started with dbLinter, I recommend watching Philipp Harenfeller’s video. United Codes and Grisselbav are collaborating on this product.

Feedback is very much appreciated. Comment on this blog post or raise an issue on the dbLinter GitHub repository.

Thanks.

The post Outer Joining With ANSI SQL-89 and SQL-92 appeared first on Philipp Salvisberg's Blog.

]]>