SQL Archives - Philipp Salvisberg's Blog https://www.salvis.com/blog/tag/sql/ Database-centric development Sat, 29 Aug 2026 07:52:43 +0000 en-US hourly 1 https://wordpress.org/?v=7.1.2 https://www.salvis.com/blog/wp-content/uploads/2014/04/favicon.png SQL Archives - Philipp Salvisberg's Blog https://www.salvis.com/blog/tag/sql/ 32 32 Implicit Outer Joins in Join-to-One: Feature or Flaw? https://www.salvis.com/blog/2026/08/29/implicit-outer-joins-in-join-to-one-feature-or-flaw/ https://www.salvis.com/blog/2026/08/29/implicit-outer-joins-in-join-to-one-feature-or-flaw/#respond Sat, 29 Aug 2026 07:52:40 +0000 https://www.salvis.com/blog/?p=23948 Background As you probably know, I’m maintaining a parser for grammars used in SQL files for Oracle AI Database and PostgreSQL. As a result, I check for new statements and clauses in every version of those DBMSs. PostgreSQL typically adds new features only in major versions; this means yearly. Oracle, on the […]

The post Implicit Outer Joins in Join-to-One: Feature or Flaw? appeared first on Philipp Salvisberg's Blog.

]]> Background

As you probably know, I’m maintaining a parser for grammars used in SQL files for Oracle AI Database and PostgreSQL. As a result, I check for new statements and clauses in every version of those DBMSs. PostgreSQL typically adds new features only in major versions; this means yearly. Oracle, on the other hand, documents new features in every new RU; this means quarterly.

The RU for 26.2 was a bit delayed. So, on 1 May 2026, I went through the new features in 26.2 and discovered the join-to-one clause while documenting the changes required for IslandSQL in a GitHub issue. I tried the feature in an OCI instance and posted about it on Bluesky.

Join to one post on Bluesky

This led to some discussions in private channels.

Back then, I hadn’t made up my mind regarding the usefulness of this feature, but the fact that a join uses outer-join semantics by default irritated me from the very first moment.

In the meantime, I believe that this feature is useful for various use cases. I’m not going to talk about that in this blog post. However, the longer I thought about the implicit outer join in join-to-one, the more it irritated me.

Join-To-One Will Make It Into the SQL Standard

According to Peter Eisentraut’s blog post, this feature has been accepted into the next SQL standard and is now part of its working draft. I wonder whether they also discussed the default join behaviour of a join-to-one and what the consensus was.

Anyway, since version 26.2, this feature is part of Oracle AI Database. Changing the behaviour at that stage is only feasible with a database parameter similar to group_by_position_enabled to keep backward compatibility. I doubt that something like this will happen.

What Is the Problem?

It is the first time an outer join has become the default. For every other join variant since ANSI SQL-86, an inner join has been the default. We are used to that.

Changing such a long-established convention is not necessarily a bad thing. However, there should be a convincing reason for doing so.

The SQL Language Reference provides this reason:

“The database chose this default because INNER JOINs often filter rows unintentionally.”

So, the intention is to avoid accidental filtering. That sounds reasonable. But is silently preserving rows really better than silently filtering them?

In both cases, the omitted join type affects the result. An implicit inner join may remove rows. An implicit outer join may introduce null values that have to be considered in subsequent expressions and predicates.

In any case, the join-to-one clause relies on integrity constraints. These constraints are already used to derive the join condition. So why not consider them fully and choose an inner join for mandatory foreign keys and an outer join for optional foreign keys?

Of course, this would make the join type depend on the schema definition. A change to the optionality of a foreign key could then change the semantics of an existing query. That would have been a very good reason not to choose a fixed default join type, right?

However, while the optimiser could deal with a dynamically determined join type, the query would be more challenging to read and understand. We would need to check the current optionality of each foreign key involved to determine whether nulls need to be handled.

Ouch. This would be cumbersome without implementing an additional concept that forces us to handle null values in SQL expressions and predicates (something similar to JSpecify). Therefore, it’s probably better not to derive the join type automatically from a foreign key’s optionality. Being explicit has its value.

Furthermore, an outer join limits the optimiser’s solution space and can affect performance, as Andrej Pashchenko demonstrated for join-to-one in this blog post.

These performance aspects may be addressed in upcoming versions. But the unfamiliar implicit default join strategy remains.

An Example

Let’s compare the following two queries:

1) Implicit outer joins
select count(*)
  from employees e
  join to one (
          departments d,
          jobs j,
          employees mgr on e.manager_id 
                       = mgr.employee_id
       )
 where e.salary > mgr.salary;
2) Explict inner and outer joins
select count(*)
  from employees e
  join to one (
          outer join departments d
          inner join jobs j
          outer join employees mgr on e.manager_id 
                                  = mgr.employee_id
       )
 where e.salary > mgr.salary;

The queries are based on the HR example schema and produce the same result.

Which one is easier to read and understand?

Which one reveals information that might be helpful when crafting or reviewing the WHERE clause?

IMO, the second example. For both questions.

The second example shows that:

  • Employees do not necessarily belong to a department. Yes, the department_id column in the employees table is optional.
  • Not every employee has a manager.
  • Every employee has a job.

This information helps us filter data using the department or manager columns. We have to handle null values there.

Based on that, we can see a flaw in the query. We can either use an inner join for managers because we are not interested in counting employees without a manager, or we have to address employees without a manager in the WHERE clause.

So, we should change the query to something like the following:

2a) Excluding emps without mgr
select count(*)
  from employees e
  join to one (
          outer join departments d
          inner join jobs j
          inner join employees mgr on e.manager_id 
                                  = mgr.employee_id
       )
 where e.salary > mgr.salary;
2b) Including emps without mgr
select count(*)
  from employees e
  join to one (
          outer join departments d
          inner join jobs j
          outer join employees mgr on e.manager_id 
                                  = mgr.employee_id
       )
 where e.salary > mgr.salary
    or mgr.employee_id is null;

Query 2a produces the same result as queries 1 and 2, but the inner join on employees makes it clear that excluding employees without a manager was intended.

Query 2b produces additional results for employees without a manager. The outer join on employees and the null handling in the WHERE clause make it clear that this was intentional.

Summary

IMO, the implicit outer joins in the new join-to-one clause are clearly a flaw. You should always specify inner or outer joins explicitly in a join-to-one clause. This communicates the intended semantics and highlights where downstream null handling may be required.

If you are using dbLinter – which you should – then you can enable the rule G-3194 to improve the maintainability of your join-to-one clauses.

The post Implicit Outer Joins in Join-to-One: Feature or Flaw? appeared first on Philipp Salvisberg's Blog.

]]> https://www.salvis.com/blog/2026/08/29/implicit-outer-joins-in-join-to-one-feature-or-flaw/feed/ 0 The Checkbox That Disables TLS https://www.salvis.com/blog/2026/08/23/the-checkbox-that-disables-tls/ https://www.salvis.com/blog/2026/08/23/the-checkbox-that-disables-tls/#respond Sun, 23 Aug 2026 21:20:28 +0000 https://www.salvis.com/blog/?p=23591 Introduction The dbLinter clients include a new setting named Allow Insecure TLS. It is disabled by default. As a result, all REST API calls verify the TLS certificate and hostname. The setting allows dbLinter to work in organisations that inspect TLS traffic using tools such as Zscaler. In larger organisations, arranging an […]

The post The Checkbox That Disables TLS appeared first on Philipp Salvisberg's Blog.

]]> Introduction

The dbLinter clients include a new setting named Allow Insecure TLS. It is disabled by default. As a result, all REST API calls verify the TLS certificate and hostname.

The setting allows dbLinter to work in organisations that inspect TLS traffic using tools such as Zscaler.

In larger organisations, arranging an inspection bypass can be cumbersome and is often too much effort for an ad-hoc trial of dbLinter. The checkbox provides a temporary workaround and should be enabled only on a trusted network.

However, allowing insecure TLS on untrusted networks, such as open Wi-Fi without client isolation, can let an attacker redirect traffic through their own machine using techniques like ARP spoofing. The attacker can also operate the access point themselves, for example, by creating a rogue Wi-Fi hotspot.

In this blog post, I demonstrate the effect of this checkbox.

Allow Insecure TLS

The latest version of dbLinter for VS Code has a checkbox that disables certificate and hostname verification.

Allow Insecure TLS Checkbox in VS Code

dbLinter’s allowInsecureTls setting is nothing unusual. Many common tools offer a similar option, including curl (--insecure), wget (--no-check-certificate), and git (--config http.sslVerify false).

Installing and Configuring a Proxy

I run this demo on macOS 26.6.2.

A simple proxy is mitmweb. We can install and configure it as follows:

brew install mitmproxy
networksetup -setsecurewebproxy "Wi-Fi" 127.0.0.1 8090
mitmweb --listen-port 8090 --web-port 8091 --allow-hosts '^api\.dblinter\.app'
[20:36:20.907] HTTP(S) proxy listening at *:8090.
[20:36:20.908] Web server listening at http://127.0.0.1:8091/?token=f83b02032aaaccd282ad3925a484fcdb

The mitmweb command starts the proxy and listens on port 8090 for TLS traffic between localhost and api.dblinter.app. All other traffic is bypassed.

The mitmweb proxy also opens the following web page:

mitmproxy after start up

Intercepting Traffic

To generate traffic, we launch VS Code and open a SQL file. This produces the following in the output panel for dbLinter:

2026-08-23 20:37:43.216 [Info ] Starting dbLinter Language Server.
2026-08-23 20:37:43.539 [Info ] connect.
2026-08-23 20:37:43.611 [Info ] initialize VSCode 1.10.0.
2026-08-23 20:37:43.646 [Info ] didOpen file:///.../dbl_client_types.sql.
2026-08-23 20:37:43.651 [Info ] AntlrCacheCoordinator initialized with parallel degree 1 and clearCacheThreshold 2048 of 16384 MB.
2026-08-23 20:37:43.671 [Warn ] TLS certificate and hostname verification are disabled for dbLinter repository API calls. Use this temporary workaround only when TLS interception cannot be bypassed.
2026-08-23 20:37:45.533 [Info ] Configuration dbLinter of tenant Grisselbav loaded with 294 check methods.
2026-08-23 20:37:45.534 [Info ] didChangeConfiguration.
2026-08-23 20:37:45.703 [Info ] tests returned 43 tests.
2026-08-23 20:37:45.796 [Info ] parseAndCheck file:///.../dbl_client_types.sql.
2026-08-23 20:37:45.910 [Info ] hasFeature Config returned true.
2026-08-23 20:37:46.286 [Info ] parseAndCheck completed in 0.489 sec including 0.052 sec for checks.
2026-08-23 20:37:46.289 [Info ] diagnostics for file:///.../dbl_client_types.sql with 2 issues.

The log contains a warning that TLS certificate and hostname verification are disabled.

dbLinter analyses SQL and APEXlang files locally. The captured traffic lets us verify what is sent to the dbLinter API. It contains the API key and session data, but not the contents of the analysed files. The source code therefore remains within the local network.

mitmproxy request message

It goes without saying that it is not good to expose the API key in plain text.

The proxy also shows the decrypted response:

mitmproxy response message

The session object in the response contains a signature. The client can therefore detect changes to the signed content, including the validators. This prevents the proxy from silently modifying the validators or injecting malicious code. However, the signature does not prevent the proxy from reading the request and response.

Enforce TLS Verification

Now let’s uncheck the Allow Insecure TLS checkbox and execute the Reload Window command.

disallow insecure TLS in VS Code

This produces the following in the output panel for dbLinter:

2026-08-23 20:52:37.281 [Info ] Starting dbLinter Language Server.
2026-08-23 20:52:37.550 [Info ] connect.
2026-08-23 20:52:37.606 [Info ] initialize VSCode 1.10.0.
2026-08-23 20:52:38.047 [Info ] didOpen file:///.../dbl_client_types.sql.
2026-08-23 20:52:38.049 [Info ] AntlrCacheCoordinator initialized with parallel degree 1 and clearCacheThreshold 2048 of 16384 MB.
2026-08-23 20:52:38.375 [Error] Failed to load configuration.

org.springframework.web.client.ResourceAccessException: I/O error on POST request for "https://api.dblinter.app/api/client-session/open": (certificate_unknown) PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

The proxy presented a certificate for api.dblinter.app which was not issued by a trusted certification authority.

The proxy console therefore shows the following log entry:

[20:52:38.217][127.0.0.1:60439] server connect api.dblinter.app:443 (91.98.90.2:443)
[20:52:38.360][127.0.0.1:60441] client connect
[20:52:38.375][127.0.0.1:60439] Client TLS handshake failed. The client does not trust the proxy's certificate for api.dblinter.app (OpenSSL Error([('SSL routines', '', 'ssl/tls alert certificate unknown')]))
[20:52:38.377][127.0.0.1:60439] client disconnect
[20:52:38.380][127.0.0.1:60439] server disconnect api.dblinter.app:443 (91.98.90.2:443)

The REST API call failed before the proxy received the HTTP request. Therefore, the API key and other request data were not exposed.

Reset Network Configuration and Uninstall Proxy

Let’s stop the proxy by pressing Ctrl-C in the terminal window running it. Then run the following:

networksetup -setsecurewebproxystate "Wi-Fi" off
brew uninstall mitmproxy
rm -rf ~/.mitmproxy

Conclusion

The Allow Insecure TLS option solves a practical problem in environments where TLS inspection cannot easily be bypassed. However, it disables certificate and hostname verification. A proxy or an attacker controlling the network can then read requests, including the API key.

The captured traffic also confirms that dbLinter analyses SQL and APEXlang files locally. Signed server responses protect the downloaded configuration from manipulation, but they do not protect confidential request data.

Therefore, enable this option only as a temporary workaround on a trusted network. The preferred solution is to configure a TLS inspection bypass. Disable the option again as soon as possible.

The post The Checkbox That Disables TLS appeared first on Philipp Salvisberg's Blog.

]]> https://www.salvis.com/blog/2026/08/23/the-checkbox-that-disables-tls/feed/ 0 Using SQL Assertions to Enforce Temporal Data Integrity https://www.salvis.com/blog/2026/03/09/using-sql-assertions-to-enforce-temporal-data-integrity/ Mon, 09 Mar 2026 14:45:31 +0000 https://www.salvis.com/blog/?p=16264 Introduction More than twelve years ago, I described the multi-temporal features of Oracle Database in this blog post. In the conclusion, I wrote that I missed a temporal DML API, temporal integrity constraints, temporal joins, and temporal aggregations. These features are still missing today. However, in the latest version of Oracle AI […]

The post Using SQL Assertions to Enforce Temporal Data Integrity appeared first on Philipp Salvisberg's Blog.

]]> Introduction

More than twelve years ago, I described the multi-temporal features of Oracle Database in this blog post. In the conclusion, I wrote that I missed a temporal DML API, temporal integrity constraints, temporal joins, and temporal aggregations. These features are still missing today.

However, in the latest version of Oracle AI Database (23.26.1), SQL assertions make it possible to define temporal integrity constraints manually. In this post, I demonstrate how.

Data Model

There are various ways to model temporal data. In this post, I use the well-known non-temporal DEPT-EMP model as a basis and add a history table for each non-temporal table with a valid-time period named vt using the column vt_start for the beginning of the period and vt_end for the end of the period. Please note that Oracle Database uses half-open intervals for temporal periods and NULL for -∞ and +∞.

Non-temporal tables store the most recent version of the data, which is kept in their history tables. This simplifies access to the latest version and provides the expected performance for non-temporal data. The disadvantage is that the API must maintain redundant data. Note that the latest version is not necessarily the current version. The current version is the one whose valid-time period contains the current date and time.

This model versions all columns of the base table, potentially resulting in more rows in the history tables than necessary. Furthermore, the model does not support soft deletes. In other words, there is no is_deleted flag. This makes managing data integrity easier. No need to keep reference and structure data just because some deleted rows still reference it. In any case, this is sufficient for this blog post.

Setup

For the examples in this blog post, I used Oracle AI Database 26ai Enterprise Edition Release 23.26.1.0.0.

Connected as SYS, I ran the following script to create the user DEMO with the necessary privileges.

1) DEMO user
create user demo identified by demo
   default tablespace users
   temporary tablespace temp
   quota unlimited on users;
   
grant create session to demo;
grant create table to demo;
grant create sequence to demo;
grant create assertion to demo;
User DEMO created.


Grant succeeded.


Grant succeeded.


Grant succeeded.


Grant succeeded.

Connected as DEMO, I ran the following script to implement the data model as outlined in the previous chapter with initial data.

2) Install model with data
create table dept (
   deptno      number(2, 0)      not null,
   dname       varchar2(14 char) not null,
   loc         varchar2(13 char) not null,
   constraint dept_pk primary key (deptno) deferrable initially immediate
);

create table dept_ht (
   hist_id     integer generated always as identity not null,
   vt_start    date              null,
   vt_end      date              null,
   deptno      number(2, 0)      not null,
   dname       varchar2(14 char) not null,
   loc         varchar2(13 char) not null,
   period for vt (vt_start, vt_end),
   constraint dept_ht_pk primary key (hist_id),
   constraint dept_ht_uk unique (deptno, vt_start) deferrable initially immediate,
   constraint dept_ht_dept_fk foreign key (deptno) references dept deferrable initially immediate,
   constraint dept_ht_vt_ck check (vt_start is null or vt_end is null or vt_start < vt_end) deferrable initially immediate
);

create index dept_ht_dept_fk_i on dept_ht (deptno);

create table emp (
   empno       number(4, 0)      not null,
   ename       varchar2(10 char) not null,
   job         varchar2(9 char)  not null,
   mgr         number(4, 0),
   hiredate    date              not null,
   sal         number(7, 2)      not null,
   comm        number(7, 2),
   deptno      number(2, 0)      not null,
   constraint emp_pk primary key (empno) deferrable initially immediate,
   constraint emp_emp_fk foreign key (mgr) references emp deferrable initially immediate,
   constraint emp_dept_fk foreign key (deptno) references dept deferrable initially immediate
);

create index emp_dept_fk_i on emp(deptno);
create index emp_emp_fk_i on emp(mgr);

create table emp_ht (
   hist_id     integer generated always as identity not null,
   vt_start    date              null,
   vt_end      date              null,
   empno       number(4, 0)      not null,
   ename       varchar2(10 char) not null,
   job         varchar2(9 char)  not null,
   mgr         number(4, 0)      null,
   hiredate    date              not null,
   sal         number(7, 2)      not null,
   comm        number(7, 2)      null,
   deptno      number(2, 0)      not null,
   period for vt (vt_start, vt_end),
   constraint emp_ht_pk primary key (hist_id),
   constraint emp_ht_uk unique (empno, vt_start) deferrable initially immediate,
   constraint emp_ht_emp_empno_fk foreign key (empno) references emp deferrable initially immediate,
   constraint emp_ht_emp_mgr_fk foreign key (mgr) references emp deferrable initially immediate,
   constraint emp_ht_dept_fk foreign key (deptno) references dept deferrable initially immediate,
   constraint emp_ht_vt_ck check (vt_start is null or vt_end is null or vt_start < vt_end) deferrable initially immediate
);

create index emp_ht_emp_fk on emp_ht (deptno);
create index emp_ht_dept_fk on emp_ht (mgr);

insert into dept (deptno, dname, loc)
values (10, 'ACCOUNTING', 'NEW YORK'),
       (20, 'RESEARCH',   'DALLAS'),
       (30, 'SALES',      'CHICAGO'),
       (40, 'OPERATIONS', 'BOSTON');

insert into dept_ht (deptno, dname, loc)
select deptno, dname, loc
  from dept;

insert into emp (empno, ename, job, mgr, hiredate, sal, comm, deptno)
values (7566, 'JONES',  'MANAGER',   7839, date '1981-04-02', 2975, null, 20),
       (7698, 'BLAKE',  'MANAGER',   7839, date '1981-05-01', 2850, null, 30),
       (7782, 'CLARK',  'MANAGER',   7839, date '1981-06-09', 2450, null, 10),
       (7788, 'SCOTT',  'ANALYST',   7566, date '1987-04-19', 3000, null, 20),
       (7902, 'FORD',   'ANALYST',   7566, date '1981-12-03', 3000, null, 20),
       (7499, 'ALLEN',  'SALESMAN',  7698, date '1981-02-20', 1600,  300, 30),
       (7521, 'WARD',   'SALESMAN',  7698, date '1981-02-22', 1250,  500, 30),
       (7654, 'MARTIN', 'SALESMAN',  7698, date '1981-09-28', 1250, 1400, 30),
       (7844, 'TURNER', 'SALESMAN',  7698, date '1981-09-08', 1500,    0, 30),
       (7900, 'JAMES',  'CLERK',     7698, date '1981-12-03',  950, null, 30),
       (7934, 'MILLER', 'CLERK',     7782, date '1982-01-23', 1300, null, 10),
       (7369, 'SMITH',  'CLERK',     7902, date '1980-12-17',  800, null, 20),
       (7839, 'KING',   'PRESIDENT', null, date '1981-11-17', 5000, null, 10),
       (7876, 'ADAMS',  'CLERK',     7788, date '1987-05-23', 1100, null, 20);

insert into emp_ht (empno, ename, job, mgr, hiredate, sal, comm, deptno)
select empno, ename, job, mgr, hiredate, sal, comm, deptno
  from emp;

commit;
Table DEPT created.


Table DEPT_HT created.


Index DEPT_HT_DEPT_FK_I created.


Table EMP created.


Index EMP_DEPT_FK_I created.


Index EMP_EMP_FK_I created.


Table EMP_HT created.


Index EMP_HT_EMP_FK created.


Index EMP_HT_DEPT_FK created.


4 rows inserted.


4 rows inserted.


14 rows inserted.


14 rows inserted.


Commit complete.

Prohibit Overlapping Periods

Chris Saxon explains in this blog post how to prevent overlapping time ranges with SQL assertions. We use a similar approach here for dept_ht.

3) Create assertion: no overlapping in dept_ht for vt period
create assertion if not exists no_overlapping_dept_ht_vt_period_as
check (
   not exists (
      select 'a department version (d1)'
        from dept_ht d1
       where exists (
              select 'an overlapping department version (d2)'
                from dept_ht d2
               where d2.deptno = d1.deptno
                 and d2.hist_id != d1.hist_id
                 and (d1.vt_start is null or d2.vt_end is null or d1.vt_start < d2.vt_end)
                 and (d2.vt_start is null or d1.vt_end is null or d2.vt_start < d1.vt_end)
             )
   )
) deferrable initially immediate;
Assertion NO_OVERLAPPING_DEPT_HT_VT_PERIOD_AS created.

Let’s test the assertion. We defer all constraints to make the example easier to implement and read.

4) Overlapping test
set constraints all deferred;
delete from dept_ht where deptno = 10;
insert into dept_ht (deptno, dname, loc, vt_start, vt_end)
values (10, 'ACCOUNTING', 'NEW_YORK',   null,              date '2026-04-01'),
       (10, 'ACCOUNTING', 'WINTERTHUR', date '2026-03-01', null             );
commit;
Constraints ALL succeeded.


1 row deleted.


2 rows inserted.


Error starting at line : 6 in command -
commit
Error report -
ORA-02091: transaction rolled back
ORA-08601: SQL assertion (DEMO.NO_OVERLAPPING_DEPT_HT_VT_PERIOD_AS) violated.

https://docs.oracle.com/error-help/db/ora-02091/
02091. 00000 -  "transaction rolled back"
*Cause:    Also see error 2092. If the transaction is aborted at a remote
           site then you will only see 2091; if aborted at host then you will
           see 2092 and 2091.
*Action:   Add rollback segment and retry the transaction.

The test fails at line 6, at the commit statement, exactly as expected.

Let’s insert two adjacent (gapless) periods.

5) Non-overlapping test
set constraints all deferred;
delete from dept_ht where deptno = 10;
insert into dept_ht (deptno, dname, loc, vt_start, vt_end)
values (10, 'ACCOUNTING', 'NEW_YORK',   null,              date '2026-04-01'),
       (10, 'ACCOUNTING', 'WINTERTHUR', date '2026-04-01', null             );
commit;
Constraints ALL succeeded.


1 row deleted.


2 rows inserted.


Commit complete.

The test succeeds. So far so good.

However, this change leaves the redundant current-state copy in dept out of sync. After any change to dept_ht, the API should execute a statement like the following within the same transaction:

6) Updating redundancy in DEPT from DEPT_HT
update dept d
   set d.dname = ht.dname,
       d.loc   = ht.loc
  from (
          select deptno, dname, loc
            from dept_ht
           where deptno = 10
           order by vt_start desc nulls first
           fetch first 1 row only
       ) ht
 where d.deptno = ht.deptno;
1 row updated.

Of course, we would need a similar SQL assertion for the period in emp_ht. However, this is not essential for this blog post. Therefore, we will skip it.

Temporal Foreign Keys

Let’s now look at another typical problem in temporal data models: Temporal foreign keys. We have two temporal foreign keys in our model:

  • emp_ht_emp_mgr_fk
  • emp_ht_dept_fk

Although the foreign key technically references a non-temporal table, the business rule is temporal: the referenced entity must be valid throughout the relevant period of the referencing row.

In this example, that means a manager must exist for the entire validity period of the employee version.

Sounds complicated? Let’s look at an example based on emp_ht_emp_mgr_fk.

7) Inconsistent emp_ht_emp_mgr_fk
update emp_ht
   set vt_start = hiredate;

select e.vt_start, e.vt_end, e.empno, e.ename, e.mgr, m.ename as mgr_name, m.hiredate as mgr_hiredate,
       e.mgr is not null and hm.empno is null as is_inconsistent
 from emp_ht e
 left join emp m
   on m.empno = e.mgr
 left join emp_ht hm
   on hm.empno = e.mgr
      and (hm.vt_start is null or (e.vt_start is not null and hm.vt_start <= e.vt_start))
      and (hm.vt_end is null or e.vt_start is null or hm.vt_end > e.vt_start)
order by e.empno;

rollback;
14 rows updated.


VT_START   VT_END          EMPNO ENAME             MGR MGR_NAME   MGR_HIREDATE IS_INCONSISTENT
---------- ---------- ---------- ---------- ---------- ---------- ------------ ---------------
1980-12-17                  7369 SMITH            7902 FORD       1981-12-03   true           
1981-02-20                  7499 ALLEN            7698 BLAKE      1981-05-01   true           
1981-02-22                  7521 WARD             7698 BLAKE      1981-05-01   true           
1981-04-02                  7566 JONES            7839 KING       1981-11-17   true           
1981-09-28                  7654 MARTIN           7698 BLAKE      1981-05-01   false          
1981-05-01                  7698 BLAKE            7839 KING       1981-11-17   true           
1981-06-09                  7782 CLARK            7839 KING       1981-11-17   true           
1987-04-19                  7788 SCOTT            7566 JONES      1981-04-02   false          
1981-11-17                  7839 KING                                          false          
1981-09-08                  7844 TURNER           7698 BLAKE      1981-05-01   false          
1987-05-23                  7876 ADAMS            7788 SCOTT      1987-04-19   false          
1981-12-03                  7900 JAMES            7698 BLAKE      1981-05-01   false          
1981-12-03                  7902 FORD             7566 JONES      1981-04-02   false          
1982-01-23                  7934 MILLER           7782 CLARK      1981-06-09   false          

14 rows selected. 


Rollback complete.

First, we set the start of the valid period for all employees according to their hire date.

Then, we queried the employees and their managers at the start of their valid time period. Employees without a manager at that time are marked with true in the result column is_inconsistent.

CLARK, for example, was hired on 9 June 1981. His manager, KING, was hired on 17 November 1981. Consequently, CLARK had no assigned manager between 9 June 1981 and 17 November 1981. This is an invalid temporal foreign key. To resolve this issue, we must either assign CLARK a manager for this period or amend KING’s valid time period, which would likely require adjusting his hire date as well.

Valid Manager At the Start of a Period

Let’s implement the check in the previous query as an SQL assertion.

8) Create assertion: cover_vt_start_in_emp_ht_emp_mgr_fk_as
create assertion if not exists cover_vt_start_in_emp_ht_emp_mgr_fk_as
check (
   not exists (
      select 'an employee version (e)'
        from emp_ht e
       where e.mgr is not null
         and not exists (
                select 'a manager version (m) covering e.vt_start'
                  from emp_ht m
                 where m.empno = e.mgr
                   and (m.vt_start is null or (e.vt_start is not null and m.vt_start <= e.vt_start))
                   and (m.vt_end is null or e.vt_start is null or m.vt_end > e.vt_start)
             )
   )
) deferrable initially immediate;
Assertion COVER_VT_START_IN_EMP_HT_EMP_MGR_FK_AS created.

Let’s try to create a gap in manager coverage by setting the vt_start to a non-null value for a manager.

9) Test gap in manager coverage (vt_start)
update emp_ht
   set vt_start = hiredate
 where ename = 'KING';
Error starting at line : 1 in command -
update emp_ht
   set vt_start = hiredate
 where ename = 'KING'
Error report -
ORA-08601: SQL assertion (DEMO.COVER_VT_START_IN_EMP_HT_EMP_MGR_FK_AS) violated.

https://docs.oracle.com/error-help/db/ora-08601/

Good. The update fails because JONES, BLAKE and CLARK would have no manager assigned at the beginning of their valid time period.

Now let’s try to set vt_start for an employee who is not a manager.

10) Test no gap in manager coverage (vt_start)
update emp_ht
   set vt_start = hiredate
 where ename = 'ADAMS';
rollback;
1 row updated.


Rollback complete.

The update succeeded because ADAMS is not a manager and, therefore, no gap in manager coverage was created.

Valid Manager At the End of a Period

The previous SQL assertion ensures that manager coverage exists at the start of a period. However, to guarantee temporal data integrity for the entire period, we need an additional SQL assertion.

11) Create assertion: cover_vt_end_in_emp_ht_emp_mgr_fk_as
create assertion if not exists cover_vt_end_in_emp_ht_emp_mgr_fk_as
check (
   not exists (
      select 'an employee version (e)'
        from emp_ht e
       where e.mgr is not null
         and exists (
                select 'a manager version (m) ending before e.vt_end'
                  from emp_ht m
                 where m.empno = e.mgr
                   and m.vt_end is not null
                   and (e.vt_start is null or m.vt_end > e.vt_start)
                   and (e.vt_end is null or m.vt_end < e.vt_end)
                   and not exists (
                          select 'a manager version (m2) covering the m.vt_end'
                            from emp_ht m2
                           where m2.empno = e.mgr
                             and m2.hist_id != m.hist_id
                             and (m2.vt_start is null or m2.vt_start <= m.vt_end)
                             and (m2.vt_end is null or m2.vt_end > m.vt_end)
                       )
             )
   )
) deferrable initially immediate;
Assertion COVER_VT_END_IN_EMP_HT_EMP_MGR_FK_AS created.

Let’s try now to set the vt_end to a non-null value of a manager.

12) Test gap in manager coverage (vt_end)
update emp_ht
   set vt_end = date '2100-01-01'
 where ename = 'KING';
Error starting at line : 1 in command -
update emp_ht
   set vt_end = date '2100-01-01'
 where ename = 'KING'
Error at Command Line : 1 Column : 8
Error report -
SQL Error: ORA-08601: SQL assertion (DEMO.COVER_VT_END_IN_EMP_HT_EMP_MGR_FK_AS) violated.

https://docs.oracle.com/error-help/db/ora-08601/

Good. The update failed because JONES, BLAKE and CLARK would have had no manager from 1 January 2100 until the end of time.

13) Test no gap in manager coverage (vt_end)
update emp_ht
   set vt_end = date '2100-01-01'
  where ename = 'ADAMS';
rollback;
1 row updated.


Rollback complete.

The update succeeded because ADAMS is not a manager and, therefore, no gap in manager coverage was created.

Well, those were just simple test cases. What would happen if we left a one-day gap between versions of the history? Would this be detected?

14) Test gap in manager coverage of multiple versions
set constraints all deferred;
delete from emp_ht where ename = 'KING';
insert into emp_ht (vt_start, vt_end, empno, ename, job, hiredate, sal, deptno)
values (null,              date '1981-11-17', 7839, 'KING', 'PRESIDENT', date '1981-11-17', 2000, 10),
       (date '1981-11-17', date '2100-01-01', 7839, 'KING', 'PRESIDENT', date '1981-11-17', 4500, 10),
       (date '2100-01-02', date '2200-01-01', 7839, 'KING', 'PRESIDENT', date '1981-11-17', 4600, 10),
       (date '2200-01-01', null             , 7839, 'KING', 'PRESIDENT', date '1981-11-17', 5000, 10);
commit;
Constraints ALL succeeded.


1 row deleted.


4 rows inserted.


Error starting at line : 8 in command -
commit
Error report -
ORA-02091: transaction rolled back
ORA-08601: SQL assertion (DEMO.COVER_VT_END_IN_EMP_HT_EMP_MGR_FK_AS) violated.

https://docs.oracle.com/error-help/db/ora-02091/
02091. 00000 -  "transaction rolled back"
*Cause:    Also see error 2092. If the transaction is aborted at a remote
           site then you will only see 2091; if aborted at host then you will
           see 2092 and 2091.
*Action:   Add rollback segment and retry the transaction.

Yes! The commit statement failed because no version covered 1 January 2100.

When we fix the date on line 6, the transaction should succeed. Let’s try that.

15) Test no gap in manager coverage of multiple versions
set constraints all deferred;
delete from emp_ht where ename = 'KING';
insert into emp_ht (vt_start, vt_end, empno, ename, job, hiredate, sal, deptno)
values (null,              date '1981-11-17', 7839, 'KING', 'PRESIDENT', date '1981-11-17', 2000, 10),
       (date '1981-11-17', date '2100-01-01', 7839, 'KING', 'PRESIDENT', date '1981-11-17', 4500, 10),
       (date '2100-01-01', date '2200-01-01', 7839, 'KING', 'PRESIDENT', date '1981-11-17', 4600, 10),
       (date '2200-01-01', null             , 7839, 'KING', 'PRESIDENT', date '1981-11-17', 5000, 10);
commit;
Constraints ALL succeeded.


1 row deleted.


4 rows inserted.


Commit complete.

Perfect. This works.

We have successfully enforced temporal referential integrity for the manager relationship using SQL assertions.

Summary

In this blog post, we created the SQL assertion no_overlapping_dept_ht_vt_period_as to ensure validity periods in the table dept_ht do not overlap.

We enforced temporal referential data integrity for the foreign key emp_ht_emp_mgr_fk using the SQL assertions cover_vt_start_in_emp_ht_emp_mgr_fk_as and cover_vt_end_in_emp_ht_emp_mgr_fk_as. This means that an employee always has a valid manager assigned throughout their validity period. Any gaps in manager coverage are detected correctly.

Adapting these SQL assertions for use in templates for temporal API generators should not be too difficult. All necessary information is in the Oracle data dictionary. With some additional metadata, we can generate additional SQL assertions for related, optional topics such as:

  • Enforce adjacent periods (no gaps)
  • Disallow adjacent periods with the same business values (enforce merge of periods)
  • Disallow different business keys across periods (enforce business key updates on all versions)
  • Disallow new versions for changes to non-versioned columns only (enforce update of all versions instead)

Although SQL assertions make it easier to enforce the integrity of temporal data, the use of temporal tables measurably increases complexity. Therefore, only use temporal tables if you really need to. Document which columns require temporal handling and explain why.

In any case, the integration of SQL assertions into Oracle AI Database makes the use of temporal tables much easier.

The post Using SQL Assertions to Enforce Temporal Data Integrity appeared first on Philipp Salvisberg's Blog.

]]>