Valid Time Archives - Philipp Salvisberg's Blog https://www.salvis.com/blog/tag/valid-time/ Database-centric development Tue, 10 Mar 2026 10:50:01 +0000 en-US hourly 1 https://wordpress.org/?v=6.9.4 https://www.salvis.com/blog/wp-content/uploads/2014/04/favicon.png Valid Time Archives - Philipp Salvisberg's Blog https://www.salvis.com/blog/tag/valid-time/ 32 32 Using SQL Assertions to Enforce Temporal Data Integrity https://www.salvis.com/blog/2026/03/09/using-sql-assertions-to-enforce-temporal-data-integrity/ https://www.salvis.com/blog/2026/03/09/using-sql-assertions-to-enforce-temporal-data-integrity/#respond Mon, 09 Mar 2026 14:45:31 +0000 https://www.salvis.com/blog/?p=16264 Introduction More than twelve years ago, I described the multi-temporal features of Oracle Database in this blog post. In the conclusion, I wrote that I missed a temporal DML API, temporal integrity constraints, temporal joins, and temporal aggregations. These features are still missing today. However, in the latest version of Oracle AI […]

The post Using SQL Assertions to Enforce Temporal Data Integrity appeared first on Philipp Salvisberg's Blog.

]]> Introduction

More than twelve years ago, I described the multi-temporal features of Oracle Database in this blog post. In the conclusion, I wrote that I missed a temporal DML API, temporal integrity constraints, temporal joins, and temporal aggregations. These features are still missing today.

However, in the latest version of Oracle AI Database (23.26.1), SQL assertions make it possible to define temporal integrity constraints manually. In this post, I demonstrate how.

Data Model

There are various ways to model temporal data. In this post, I use the well-known non-temporal DEPT-EMP model as a basis and add a history table for each non-temporal table with a valid-time period named vt using the column vt_start for the beginning of the period and vt_end for the end of the period. Please note that Oracle Database uses half-open intervals for temporal periods and NULL for -∞ and +∞.

Non-temporal tables store the most recent version of the data, which is kept in their history tables. This simplifies access to the latest version and provides the expected performance for non-temporal data. The disadvantage is that the API must maintain redundant data. Note that the latest version is not necessarily the current version. The current version is the one whose valid-time period contains the current date and time.

This model versions all columns of the base table, potentially resulting in more rows in the history tables than necessary. Furthermore, the model does not support soft deletes. In other words, there is no is_deleted flag. This makes managing data integrity easier. No need to keep reference and structure data just because some deleted rows still reference it. In any case, this is sufficient for this blog post.

Setup

For the examples in this blog post, I used Oracle AI Database 26ai Enterprise Edition Release 23.26.1.0.0.

Connected as SYS, I ran the following script to create the user DEMO with the necessary privileges.

1) DEMO user
create user demo identified by demo
   default tablespace users
   temporary tablespace temp
   quota unlimited on users;
   
grant create session to demo;
grant create table to demo;
grant create sequence to demo;
grant create assertion to demo;
User DEMO created.


Grant succeeded.


Grant succeeded.


Grant succeeded.


Grant succeeded.

Connected as DEMO, I ran the following script to implement the data model as outlined in the previous chapter with initial data.

2) Install model with data
create table dept (
   deptno      number(2, 0)      not null,
   dname       varchar2(14 char) not null,
   loc         varchar2(13 char) not null,
   constraint dept_pk primary key (deptno) deferrable initially immediate
);

create table dept_ht (
   hist_id     integer generated always as identity not null,
   vt_start    date              null,
   vt_end      date              null,
   deptno      number(2, 0)      not null,
   dname       varchar2(14 char) not null,
   loc         varchar2(13 char) not null,
   period for vt (vt_start, vt_end),
   constraint dept_ht_pk primary key (hist_id),
   constraint dept_ht_uk unique (deptno, vt_start) deferrable initially immediate,
   constraint dept_ht_dept_fk foreign key (deptno) references dept deferrable initially immediate,
   constraint dept_ht_vt_ck check (vt_start is null or vt_end is null or vt_start < vt_end) deferrable initially immediate
);

create index dept_ht_dept_fk_i on dept_ht (deptno);

create table emp (
   empno       number(4, 0)      not null,
   ename       varchar2(10 char) not null,
   job         varchar2(9 char)  not null,
   mgr         number(4, 0),
   hiredate    date              not null,
   sal         number(7, 2)      not null,
   comm        number(7, 2),
   deptno      number(2, 0)      not null,
   constraint emp_pk primary key (empno) deferrable initially immediate,
   constraint emp_emp_fk foreign key (mgr) references emp deferrable initially immediate,
   constraint emp_dept_fk foreign key (deptno) references dept deferrable initially immediate
);

create index emp_dept_fk_i on emp(deptno);
create index emp_emp_fk_i on emp(mgr);

create table emp_ht (
   hist_id     integer generated always as identity not null,
   vt_start    date              null,
   vt_end      date              null,
   empno       number(4, 0)      not null,
   ename       varchar2(10 char) not null,
   job         varchar2(9 char)  not null,
   mgr         number(4, 0)      null,
   hiredate    date              not null,
   sal         number(7, 2)      not null,
   comm        number(7, 2)      null,
   deptno      number(2, 0)      not null,
   period for vt (vt_start, vt_end),
   constraint emp_ht_pk primary key (hist_id),
   constraint emp_ht_uk unique (empno, vt_start) deferrable initially immediate,
   constraint emp_ht_emp_empno_fk foreign key (empno) references emp deferrable initially immediate,
   constraint emp_ht_emp_mgr_fk foreign key (mgr) references emp deferrable initially immediate,
   constraint emp_ht_dept_fk foreign key (deptno) references dept deferrable initially immediate,
   constraint emp_ht_vt_ck check (vt_start is null or vt_end is null or vt_start < vt_end) deferrable initially immediate
);

create index emp_ht_emp_fk on emp_ht (deptno);
create index emp_ht_dept_fk on emp_ht (mgr);

insert into dept (deptno, dname, loc)
values (10, 'ACCOUNTING', 'NEW YORK'),
       (20, 'RESEARCH',   'DALLAS'),
       (30, 'SALES',      'CHICAGO'),
       (40, 'OPERATIONS', 'BOSTON');

insert into dept_ht (deptno, dname, loc)
select deptno, dname, loc
  from dept;

insert into emp (empno, ename, job, mgr, hiredate, sal, comm, deptno)
values (7566, 'JONES',  'MANAGER',   7839, date '1981-04-02', 2975, null, 20),
       (7698, 'BLAKE',  'MANAGER',   7839, date '1981-05-01', 2850, null, 30),
       (7782, 'CLARK',  'MANAGER',   7839, date '1981-06-09', 2450, null, 10),
       (7788, 'SCOTT',  'ANALYST',   7566, date '1987-04-19', 3000, null, 20),
       (7902, 'FORD',   'ANALYST',   7566, date '1981-12-03', 3000, null, 20),
       (7499, 'ALLEN',  'SALESMAN',  7698, date '1981-02-20', 1600,  300, 30),
       (7521, 'WARD',   'SALESMAN',  7698, date '1981-02-22', 1250,  500, 30),
       (7654, 'MARTIN', 'SALESMAN',  7698, date '1981-09-28', 1250, 1400, 30),
       (7844, 'TURNER', 'SALESMAN',  7698, date '1981-09-08', 1500,    0, 30),
       (7900, 'JAMES',  'CLERK',     7698, date '1981-12-03',  950, null, 30),
       (7934, 'MILLER', 'CLERK',     7782, date '1982-01-23', 1300, null, 10),
       (7369, 'SMITH',  'CLERK',     7902, date '1980-12-17',  800, null, 20),
       (7839, 'KING',   'PRESIDENT', null, date '1981-11-17', 5000, null, 10),
       (7876, 'ADAMS',  'CLERK',     7788, date '1987-05-23', 1100, null, 20);

insert into emp_ht (empno, ename, job, mgr, hiredate, sal, comm, deptno)
select empno, ename, job, mgr, hiredate, sal, comm, deptno
  from emp;

commit;
Table DEPT created.


Table DEPT_HT created.


Index DEPT_HT_DEPT_FK_I created.


Table EMP created.


Index EMP_DEPT_FK_I created.


Index EMP_EMP_FK_I created.


Table EMP_HT created.


Index EMP_HT_EMP_FK created.


Index EMP_HT_DEPT_FK created.


4 rows inserted.


4 rows inserted.


14 rows inserted.


14 rows inserted.


Commit complete.

Prohibit Overlapping Periods

Chris Saxon explains in this blog post how to prevent overlapping time ranges with SQL assertions. We use a similar approach here for dept_ht.

3) Create assertion: no overlapping in dept_ht for vt period
create assertion if not exists no_overlapping_dept_ht_vt_period_as
check (
   not exists (
      select 'a department version (d1)'
        from dept_ht d1
       where exists (
              select 'an overlapping department version (d2)'
                from dept_ht d2
               where d2.deptno = d1.deptno
                 and d2.hist_id != d1.hist_id
                 and (d1.vt_start is null or d2.vt_end is null or d1.vt_start < d2.vt_end)
                 and (d2.vt_start is null or d1.vt_end is null or d2.vt_start < d1.vt_end)
             )
   )
) deferrable initially immediate;
Assertion NO_OVERLAPPING_DEPT_HT_VT_PERIOD_AS created.

Let’s test the assertion. We defer all constraints to make the example easier to implement and read.

4) Overlapping test
set constraints all deferred;
delete from dept_ht where deptno = 10;
insert into dept_ht (deptno, dname, loc, vt_start, vt_end)
values (10, 'ACCOUNTING', 'NEW_YORK',   null,              date '2026-04-01'),
       (10, 'ACCOUNTING', 'WINTERTHUR', date '2026-03-01', null             );
commit;
Constraints ALL succeeded.


1 row deleted.


2 rows inserted.


Error starting at line : 6 in command -
commit
Error report -
ORA-02091: transaction rolled back
ORA-08601: SQL assertion (DEMO.NO_OVERLAPPING_DEPT_HT_VT_PERIOD_AS) violated.

https://docs.oracle.com/error-help/db/ora-02091/
02091. 00000 -  "transaction rolled back"
*Cause:    Also see error 2092. If the transaction is aborted at a remote
           site then you will only see 2091; if aborted at host then you will
           see 2092 and 2091.
*Action:   Add rollback segment and retry the transaction.

The test fails at line 6, at the commit statement, exactly as expected.

Let’s insert two adjacent (gapless) periods.

5) Non-overlapping test
set constraints all deferred;
delete from dept_ht where deptno = 10;
insert into dept_ht (deptno, dname, loc, vt_start, vt_end)
values (10, 'ACCOUNTING', 'NEW_YORK',   null,              date '2026-04-01'),
       (10, 'ACCOUNTING', 'WINTERTHUR', date '2026-04-01', null             );
commit;
Constraints ALL succeeded.


1 row deleted.


2 rows inserted.


Commit complete.

The test succeeds. So far so good.

However, this change leaves the redundant current-state copy in dept out of sync. After any change to dept_ht, the API should execute a statement like the following within the same transaction:

6) Updating redundancy in DEPT from DEPT_HT
update dept d
   set d.dname = ht.dname,
       d.loc   = ht.loc
  from (
          select deptno, dname, loc
            from dept_ht
           where deptno = 10
           order by vt_start desc nulls first
           fetch first 1 row only
       ) ht
 where d.deptno = ht.deptno;
1 row updated.

Of course, we would need a similar SQL assertion for the period in emp_ht. However, this is not essential for this blog post. Therefore, we will skip it.

Temporal Foreign Keys

Let’s now look at another typical problem in temporal data models: Temporal foreign keys. We have two temporal foreign keys in our model:

  • emp_ht_emp_mgr_fk
  • emp_ht_dept_fk

Although the foreign key technically references a non-temporal table, the business rule is temporal: the referenced entity must be valid throughout the relevant period of the referencing row.

In this example, that means a manager must exist for the entire validity period of the employee version.

Sounds complicated? Let’s look at an example based on emp_ht_emp_mgr_fk.

7) Inconsistent emp_ht_emp_mgr_fk
update emp_ht
   set vt_start = hiredate;

select e.vt_start, e.vt_end, e.empno, e.ename, e.mgr, m.ename as mgr_name, m.hiredate as mgr_hiredate,
       e.mgr is not null and hm.empno is null as is_inconsistent
 from emp_ht e
 left join emp m
   on m.empno = e.mgr
 left join emp_ht hm
   on hm.empno = e.mgr
      and (hm.vt_start is null or (e.vt_start is not null and hm.vt_start <= e.vt_start))
      and (hm.vt_end is null or e.vt_start is null or hm.vt_end > e.vt_start)
order by e.empno;

rollback;
14 rows updated.


VT_START   VT_END          EMPNO ENAME             MGR MGR_NAME   MGR_HIREDATE IS_INCONSISTENT
---------- ---------- ---------- ---------- ---------- ---------- ------------ ---------------
1980-12-17                  7369 SMITH            7902 FORD       1981-12-03   true           
1981-02-20                  7499 ALLEN            7698 BLAKE      1981-05-01   true           
1981-02-22                  7521 WARD             7698 BLAKE      1981-05-01   true           
1981-04-02                  7566 JONES            7839 KING       1981-11-17   true           
1981-09-28                  7654 MARTIN           7698 BLAKE      1981-05-01   false          
1981-05-01                  7698 BLAKE            7839 KING       1981-11-17   true           
1981-06-09                  7782 CLARK            7839 KING       1981-11-17   true           
1987-04-19                  7788 SCOTT            7566 JONES      1981-04-02   false          
1981-11-17                  7839 KING                                          false          
1981-09-08                  7844 TURNER           7698 BLAKE      1981-05-01   false          
1987-05-23                  7876 ADAMS            7788 SCOTT      1987-04-19   false          
1981-12-03                  7900 JAMES            7698 BLAKE      1981-05-01   false          
1981-12-03                  7902 FORD             7566 JONES      1981-04-02   false          
1982-01-23                  7934 MILLER           7782 CLARK      1981-06-09   false          

14 rows selected. 


Rollback complete.

First, we set the start of the valid period for all employees according to their hire date.

Then, we queried the employees and their managers at the start of their valid time period. Employees without a manager at that time are marked with true in the result column is_inconsistent.

CLARK, for example, was hired on 9 June 1981. His manager, KING, was hired on 17 November 1981. Consequently, CLARK had no assigned manager between 9 June 1981 and 17 November 1981. This is an invalid temporal foreign key. To resolve this issue, we must either assign CLARK a manager for this period or amend KING’s valid time period, which would likely require adjusting his hire date as well.

Valid Manager At the Start of a Period

Let’s implement the check in the previous query as an SQL assertion.

8) Create assertion: cover_vt_start_in_emp_ht_emp_mgr_fk_as
create assertion if not exists cover_vt_start_in_emp_ht_emp_mgr_fk_as
check (
   not exists (
      select 'an employee version (e)'
        from emp_ht e
       where e.mgr is not null
         and not exists (
                select 'a manager version (m) covering e.vt_start'
                  from emp_ht m
                 where m.empno = e.mgr
                   and (m.vt_start is null or (e.vt_start is not null and m.vt_start <= e.vt_start))
                   and (m.vt_end is null or e.vt_start is null or m.vt_end > e.vt_start)
             )
   )
) deferrable initially immediate;
Assertion COVER_VT_START_IN_EMP_HT_EMP_MGR_FK_AS created.

Let’s try to create a gap in manager coverage by setting the vt_start to a non-null value for a manager.

9) Test gap in manager coverage (vt_start)
update emp_ht
   set vt_start = hiredate
 where ename = 'KING';
Error starting at line : 1 in command -
update emp_ht
   set vt_start = hiredate
 where ename = 'KING'
Error report -
ORA-08601: SQL assertion (DEMO.COVER_VT_START_IN_EMP_HT_EMP_MGR_FK_AS) violated.

https://docs.oracle.com/error-help/db/ora-08601/

Good. The update fails because JONES, BLAKE and CLARK would have no manager assigned at the beginning of their valid time period.

Now let’s try to set vt_start for an employee who is not a manager.

10) Test no gap in manager coverage (vt_start)
update emp_ht
   set vt_start = hiredate
 where ename = 'ADAMS';
rollback;
1 row updated.


Rollback complete.

The update succeeded because ADAMS is not a manager and, therefore, no gap in manager coverage was created.

Valid Manager At the End of a Period

The previous SQL assertion ensures that manager coverage exists at the start of a period. However, to guarantee temporal data integrity for the entire period, we need an additional SQL assertion.

11) Create assertion: cover_vt_end_in_emp_ht_emp_mgr_fk_as
create assertion if not exists cover_vt_end_in_emp_ht_emp_mgr_fk_as
check (
   not exists (
      select 'an employee version (e)'
        from emp_ht e
       where e.mgr is not null
         and exists (
                select 'a manager version (m) ending before e.vt_end'
                  from emp_ht m
                 where m.empno = e.mgr
                   and m.vt_end is not null
                   and (e.vt_start is null or m.vt_end > e.vt_start)
                   and (e.vt_end is null or m.vt_end < e.vt_end)
                   and not exists (
                          select 'a manager version (m2) covering the m.vt_end'
                            from emp_ht m2
                           where m2.empno = e.mgr
                             and m2.hist_id != m.hist_id
                             and (m2.vt_start is null or m2.vt_start <= m.vt_end)
                             and (m2.vt_end is null or m2.vt_end > m.vt_end)
                       )
             )
   )
) deferrable initially immediate;
Assertion COVER_VT_END_IN_EMP_HT_EMP_MGR_FK_AS created.

Let’s try now to set the vt_end to a non-null value of a manager.

12) Test gap in manager coverage (vt_end)
update emp_ht
   set vt_end = date '2100-01-01'
 where ename = 'KING';
Error starting at line : 1 in command -
update emp_ht
   set vt_end = date '2100-01-01'
 where ename = 'KING'
Error at Command Line : 1 Column : 8
Error report -
SQL Error: ORA-08601: SQL assertion (DEMO.COVER_VT_END_IN_EMP_HT_EMP_MGR_FK_AS) violated.

https://docs.oracle.com/error-help/db/ora-08601/

Good. The update failed because JONES, BLAKE and CLARK would have had no manager from 1 January 2100 until the end of time.

13) Test no gap in manager coverage (vt_end)
update emp_ht
   set vt_end = date '2100-01-01'
  where ename = 'ADAMS';
rollback;
1 row updated.


Rollback complete.

The update succeeded because ADAMS is not a manager and, therefore, no gap in manager coverage was created.

Well, those were just simple test cases. What would happen if we left a one-day gap between versions of the history? Would this be detected?

14) Test gap in manager coverage of multiple versions
set constraints all deferred;
delete from emp_ht where ename = 'KING';
insert into emp_ht (vt_start, vt_end, empno, ename, job, hiredate, sal, deptno)
values (null,              date '1981-11-17', 7839, 'KING', 'PRESIDENT', date '1981-11-17', 2000, 10),
       (date '1981-11-17', date '2100-01-01', 7839, 'KING', 'PRESIDENT', date '1981-11-17', 4500, 10),
       (date '2100-01-02', date '2200-01-01', 7839, 'KING', 'PRESIDENT', date '1981-11-17', 4600, 10),
       (date '2200-01-01', null             , 7839, 'KING', 'PRESIDENT', date '1981-11-17', 5000, 10);
commit;
Constraints ALL succeeded.


1 row deleted.


4 rows inserted.


Error starting at line : 8 in command -
commit
Error report -
ORA-02091: transaction rolled back
ORA-08601: SQL assertion (DEMO.COVER_VT_END_IN_EMP_HT_EMP_MGR_FK_AS) violated.

https://docs.oracle.com/error-help/db/ora-02091/
02091. 00000 -  "transaction rolled back"
*Cause:    Also see error 2092. If the transaction is aborted at a remote
           site then you will only see 2091; if aborted at host then you will
           see 2092 and 2091.
*Action:   Add rollback segment and retry the transaction.

Yes! The commit statement failed because no version covered 1 January 2100.

When we fix the date on line 6, the transaction should succeed. Let’s try that.

15) Test no gap in manager coverage of multiple versions
set constraints all deferred;
delete from emp_ht where ename = 'KING';
insert into emp_ht (vt_start, vt_end, empno, ename, job, hiredate, sal, deptno)
values (null,              date '1981-11-17', 7839, 'KING', 'PRESIDENT', date '1981-11-17', 2000, 10),
       (date '1981-11-17', date '2100-01-01', 7839, 'KING', 'PRESIDENT', date '1981-11-17', 4500, 10),
       (date '2100-01-01', date '2200-01-01', 7839, 'KING', 'PRESIDENT', date '1981-11-17', 4600, 10),
       (date '2200-01-01', null             , 7839, 'KING', 'PRESIDENT', date '1981-11-17', 5000, 10);
commit;
Constraints ALL succeeded.


1 row deleted.


4 rows inserted.


Commit complete.

Perfect. This works.

We have successfully enforced temporal referential integrity for the manager relationship using SQL assertions.

Summary

In this blog post, we created the SQL assertion no_overlapping_dept_ht_vt_period_as to ensure validity periods in the table dept_ht do not overlap.

We enforced temporal referential data integrity for the foreign key emp_ht_emp_mgr_fk using the SQL assertions cover_vt_start_in_emp_ht_emp_mgr_fk_as and cover_vt_end_in_emp_ht_emp_mgr_fk_as. This means that an employee always has a valid manager assigned throughout their validity period. Any gaps in manager coverage are detected correctly.

Adapting these SQL assertions for use in templates for temporal API generators should not be too difficult. All necessary information is in the Oracle data dictionary. With some additional metadata, we can generate additional SQL assertions for related, optional topics such as:

  • Enforce adjacent periods (no gaps)
  • Disallow adjacent periods with the same business values (enforce merge of periods)
  • Disallow different business keys across periods (enforce business key updates on all versions)
  • Disallow new versions for changes to non-versioned columns only (enforce update of all versions instead)

Although SQL assertions make it easier to enforce the integrity of temporal data, the use of temporal tables measurably increases complexity. Therefore, only use temporal tables if you really need to. Document which columns require temporal handling and explain why.

In any case, the integration of SQL assertions into Oracle AI Database makes the use of temporal tables much easier.

The post Using SQL Assertions to Enforce Temporal Data Integrity appeared first on Philipp Salvisberg's Blog.

]]> https://www.salvis.com/blog/2026/03/09/using-sql-assertions-to-enforce-temporal-data-integrity/feed/ 0 Names Matter https://www.salvis.com/blog/2020/06/21/names-matter/ https://www.salvis.com/blog/2020/06/21/names-matter/#comments Sun, 21 Jun 2020 15:00:30 +0000 https://www.salvis.com/blog/?p=9890 This is one of my favourite quotes: There are only two hard things in Computer Science: cache invalidation and naming things.— Phil Karlton IT is my daily life. And this quote is so true. Lately, I’ve been thinking much more than usual about naming, and that names really matter. This led to […]

The post Names Matter appeared first on Philipp Salvisberg's Blog.

]]> This is one of my favourite quotes:

There are only two hard things in Computer Science: cache invalidation and naming things.
— Phil Karlton

IT is my daily life. And this quote is so true. Lately, I’ve been thinking much more than usual about naming, and that names really matter. This led to some refactoring activities.

Why Is Naming Important?

When a person with German mother tongue hears the word “eagle”, she or he automatically associates it with a “hedgehog”. Simply because the German word for it (“Igel”) is pronounced exactly the same. Of course, language skills and the concrete context play a role. The point is, a wrong association is likely. When we give a name to a thing, we basically want to avoid such false associations. In the best case, they are not helpful. In the worst case, this leads to rejection, as the next example shows.

In 1982 Mitsubishi Motors launched a SUV with the name “Pajero”. This name had to be changed in some regions, because “pajero” means “wanker” in Spanish. This example also shows that it is more important what others think about a name than we do.

In IT we have to name many things. Databases, schemas, tables, columns, views, packages, triggers, variables, fields, methods, classes, modules, components, products, etc. etc. Using an established name with a known and accepted definition help others to understand it better.

When we use a name, it is actually associated with a definition and properties, whether we like it or not. When names have a common and widely accepted meaning, it simplifies the communication. For example “banana”. Everybody knows what it means. Merriam-Webster’s definition is:

An elongated usually tapering tropical fruit with soft pulpy flesh enclosed in a soft usually yellow rind.

And I am sure that each of us could add a few characteristics to this definition.

Why Is Naming Difficult?

A name must fulfil many characteristics. For example

  • Short
  • Fitting (naturally relates to the intended meaning, and characteristics)
  • Easy to spell, pronounce, remember
  • Not associated with unwanted characteristics
  • Common and widely accepted meaning and definition, that fits the intention (for names without commercial value)
  • New, not used already (for marketable names)

Depending on context there are some goal conflicts. However, even without a major conflict, it is difficult to name something adequately in the early stages. Because we do not know enough about the thing we want to name. Hence, we use an iterative approach. We name something (e.g. an entity, package or class) and while working on it we find out that the name does not fit (anymore) and we change it. Maybe we split the thing and have to name now two things, etc. etc.

Finding a fitting name means doing some research. How have others named that thing? What is the definition of it? Does it fit 100 percent? This is an interesting and instructive work. In any case, it takes time. And at the time we need a new name, we want it now (e.g. when a wizard asks for a name). We can always rename it later, right? – Technically yes. And often we do. But the longer we wait, the less likely we are renaming.

Are Some Names More Important Than Others?

Yes. The more visible a name is the more important it is.

For example, the names behind an API are very easy to change. We do not have to ask anyone before changing it. It’s no problem as long as the API provides the same results. That’s one of the reasons we strive for tight APIs, right? To get some leeway.

As soon as others are involved, we are not fully in control of the change anymore. For example, when I change a name in one of my blog posts, this change is visible immediately to everyone visiting my blog. But I cannot control the caches of others, like search engines, blog mirrors and other services that copy web content to third-party storages. Remember, cache invalidation is the other hard thing in IT.

As a consequence, before we release an artefact that becomes visible to others, we should take some time to verify the used names. We cannot take back what we’ve said (at least not completely). However, we are in control of what we say in the future.

Banned Names on This Blog

Some terms (names) were discussed recently (again) due to a series of sad events. I used these terms as well. I never really thought about them as “bad”. However, I’ve changed my mind. I’m part of the problem. And I do not like it. One thing I can do is to stop using terms, that a large group of people associate with slavery and racism. No big deal, right?

This is another quote I like very much:

One cannot not communicate
— Paul Watzlawick

It is difficult to draw a line for certain terms. However, I believe that “you cannot not decide”. You decide either explicitly or implicitly. Of course, very seldom something is purely black or white. It’s much more often a shade of grey. Some decision take some time. And that’s okay. But it is impossible to postpone a decision forever. At a certain point, it becomes a decision.

So, I decided to decommission some terms on this blog and introduce new ones. Here’s the list:

Current TermDecommissioned TermContext
accessiblewhite listedPL/SQL accessible_by clause
agentslaveJenkins
exclusion listblacklistPL/SQL Cop, PL/SQL accessible_by clause
inclusion listwhitelistPL/SQL Cop, PL/SQL accessible_by clause
mainmasterGit branch
transaction structure data + enterprise structure datamaster dataData modeling
workerslaveOracle DB background process

Finding alternative names was surprisingly easy because others had already done the work and defined alternative names. They existed for years…

Master Data

However, finding an alternative for master data was harder. I reached out to my friends on Twitter. And got some helpful feedback. Finally, Robert Marti suggested having a look at Malcolm Chisholm‘s book Managing Reference Data in Enterprise Databases. On page 258ff the different data classes are defined and explained. The book is from 2000. In the meantime Malcolm Chisholm has published revised definitions here and here.

In the next subchapter, I repeat the definition of the data groups defined by Malcolm Chisholm on slide 5 in this deck. I like these definitions and plan to use them in the future.

Metadata

The data that describes all aspects of an enterprise’s information assets, and enables the enterprise to effectively use and manage these assets.

Here it is confined to the structure of databases. Found in a database’s system catalog. Sometimes included in database tables.

Reference Data

Any kind of data that is used solely to categorize other data found in a database, or solely for relating data in a database to information beyond the boundaries of the enterprise.

Codes and descriptions. Tables containing this data usually have just a few rows and columns.

Transaction Structure Data

Data that represents the direct participants in a transaction, and which must be present before a transaction fires.

The parties to the transactions of the enterprise. E.g. Customer, Product.

Enterprise Structure Data

Data that permits business activity to be reported and/or analyzed by business responsibility.

Typically, data that describes the structure of the enterprise. E.g. organizational or financial structure.

Transaction Activity Data

Data that represents the operations an enterprise carries out.

Traditional focus of IT – in many enterprises the only focus.

Transaction Audit Data

Data that tracks the life cycle of individual transactions.

Includes application logs, database logs, web server logs.

Summary

You use a name to simplify communication. A name is a proxy for a longer definition and meaning. If the meaning is badly received by others and especially by the target community, this does not simplify communication. Using a different name sounds like a simple solution. Why not, if changing a name is simple enough?

In this case, I only had to edit a few blog posts. I handled them like typos. This means that I did not add any update information. I also had to register new URL redirects. That was straightforward. However, changing the branch name in 26 GitHub repositories was a bit more work than anticipated, because I also had to change URLs in several related files. For certain GitHub pages, I had to keep a non-default master branch. I suppose that sooner or later GitHub will allow me to get rid of them as well. If I had to change more repositories, I would probably automate this task.

Most of the time I spent finding an alternative name for “master data”. In the end, I learned something new and found good names and definitions. That will help me in the future.

The post Names Matter appeared first on Philipp Salvisberg's Blog.

]]> https://www.salvis.com/blog/2020/06/21/names-matter/feed/ 2 Bitemp Remodeler v0.1.0 Released https://www.salvis.com/blog/2016/09/25/bitemp-remodeler-v0-1-0-released/ Sun, 25 Sep 2016 21:21:21 +0000 https://www.salvis.com/blog/?p=7254 I’ve been working on a flexible table API generator for Oracle Databases for several months. A TAPI generator doesn’t sound like a real innovation. But this one contains some features you probably have not seen before in the TAPI generator and hopefully will like it as much as I do. In this post, I will […]

The post Bitemp Remodeler v0.1.0 Released appeared first on Philipp Salvisberg's Blog.

]]> I’ve been working on a flexible table API generator for Oracle Databases for several months. A TAPI generator doesn’t sound like a real innovation. But this one contains some features you probably have not seen before in the TAPI generator and hopefully will like it as much as I do.

In this post, I will not explain the feature set thoroughly. Instead, I will more or less focus on one of my favourite features.

Four models

The generator knows the following four data models.

four_models

If your table is based on one of these four models you may

  1. simply generate a table API for it or
  2. switch to another model and optionally generate a table API as well.

Option 2) is extraordinary, since it will preserve the existing data. E.g. it will preserve the content of the flashback data archive when you switch your model from uni-temporal transaction time to a bi-temporal model even if the flashback archive tables need to be moved to another table. Furthermore, it will keep the interface for the latest table the same. No application change is required. Everything with just a few mouse clicks. If this sounds interesting to you, then have a look at https://github.com/oddgen/bitemp/blob/main/README.md where the concept is briefly explained or join me in my session “oddgen – Bi-temporal Table API in Action” at the More than just – Performance Days 2016. Remote participation is still possible.

Option 1) is what we had for years. It was part of Oracle Designer, it’s part of SQL Developer in a simplified way and there are some more or less simple table API generators around. So no big deal. However, when you choose option 1), there is one cool part. The hook API package concept.

The Hook API

The problem with a lot of table API solutions is, that there is typically no developer-friendly way to include the business logic. I’ve seen the following:

  • Manual changes of the generated code, which is for various reasons not a good solution.
  • External hooks, e.g. in XML files, INI files, relational tables, etc. and merged at generation time into the final code. Oracle Designer worked that way.
  • Code which is dynamically executed by the generator at runtime, e.g. code snippets is stored in a pre-defined way in relational tables.

But what I’ve never seen, was business logic implemented in manually crafted PL/SQL packages, separated from the PL/SQL generated code. That’s strange because this is a common practice in Java-based projects.

In Java, you typically define an interface for that and configure at runtime the right implementation. In PL/SQL we may do that similarly. A PL/SQL specification is an interface definition. That just one implementation may exist for an interface is not a limiting factor in this case.

Bitemp Remodeler generates the following hook API package specification for the famous EMP table in schema SCOTT:

Package specification EMP_HOOK
CREATE OR REPLACE PACKAGE emp_hook AS
   /** 
   * Hooks called by non-temporal API for table emp_lt (see package body of emp_api)
   * generated by Bitemp Remodeler for SQL Developer.
   * The body of this package is not generated. It has to be crafted and maintained manually. 
   * Since the API for table emp_lt ignores errors caused by a missing hook package body, the implementation is optional.
   *
   * @headcom
   */

   /**
   * Hook called before insert into non-temporal table emp_lt.
   *
   * @param io_new_row new Row to be inserted
   */
   PROCEDURE pre_ins (
      io_new_row IN OUT emp_ot
   );

   /**
   * Hook called after insert into non-temporal table emp_lt.
   *
   * @param in_new_row new Row to be inserted
   */
   PROCEDURE post_ins (
      in_new_row IN emp_ot
   );

   /**
   * Hook called before update non-temporal table emp_lt.
   *
   * @param io_new_row Row with updated column values
   * @param in_old_row Row with original column values
   */
   PROCEDURE pre_upd (
      io_new_row IN OUT emp_ot,
      in_old_row IN emp_ot
   );

   /**
   * Hook called after update non-temporal table emp_lt.
   *
   * @param in_new_row Row with updated column values
   * @param in_old_row Row with original column values
   */
   PROCEDURE post_upd (
      in_new_row IN emp_ot,
      in_old_row IN emp_ot
   );

   /**
   * Hook called before delete from non-temporal table emp_lt.
   *
   * @param in_old_row Row with original column values
   */
   PROCEDURE pre_del (
      in_old_row IN emp_ot
   );

   /**
   * Hook called after delete from non-temporal table emp_lt.
   *
   * @param in_old_row Row with original column values
   */
   PROCEDURE post_del (
      in_old_row IN emp_ot
   );

END emp_hook;
/

The generated table API calls before an INSERT the pre_ins procedure and after the INSERT the post_ins procedures. For DELETE and UPDATE this works the same way. On the highlighted line 5 and 6 two interested things are pointed out. The body is not generated and the body does not need to be implemented since the API ignores errors caused by a missing PL/SQL hook package body.

Technically this is solved as follows in the API package body:

Excerpt 1 of package body EMP_API
   e_hook_body_missing EXCEPTION;
   PRAGMA exception_init(e_hook_body_missing, -6508);
Excerpt 2 of package body EMP_API
      <<pre_ins>>
      BEGIN
         emp_hook.pre_ins(io_new_row => l_new_row);
      EXCEPTION
         WHEN e_hook_body_missing THEN
            NULL;
      END pre_ins;
      do_ins(io_row => l_new_row);

Now you may ask what the performance impact of these e_hook_body_missing exceptions are. I’ve done a small test and called a procedure without and with implemented body 1 million times. The overhead of the missing body exception is about 7 microseconds per call. Here’s the test output from SQL Developer, the relevant lines 51 and 89 are highlighted.

Overhead of missing package body
SQL> SET FEEDBACK ON
SQL> SET ECHO ON
SQL> SET TIMING ON
SQL> DROP PACKAGE dummy_api;

Package DUMMY_API dropped.

Elapsed: 00:00:00.027
SQL> DROP PACKAGE dummy_hook;

Package DUMMY_HOOK dropped.

Elapsed: 00:00:00.030
SQL> CREATE OR REPLACE PACKAGE dummy_hook AS
   PROCEDURE pre_ins;
END dummy_hook;
/

Package DUMMY_HOOK compiled

Elapsed: 00:00:00.023
SQL> CREATE OR REPLACE PACKAGE dummy_api AS
   PROCEDURE ins;
END dummy_api;
/

Package DUMMY_API compiled

Elapsed: 00:00:00.034
SQL> CREATE OR REPLACE PACKAGE BODY dummy_api AS
   e_hook_body_missing EXCEPTION;
   PRAGMA exception_init(e_hook_body_missing, -6508);  
   PROCEDURE ins IS
   BEGIN
      BEGIN
         dummy_hook.pre_ins;
      EXCEPTION
         WHEN e_hook_body_missing THEN
            NULL;
      END pre_ins;
      dbms_output.put('.');
   END ins;
END dummy_api;
/

Package body DUMMY_API compiled

Elapsed: 00:00:00.040
SQL> -- without hook body
SQL> BEGIN
   FOR i IN 1..1E6 LOOP
      dummy_api.ins;
   END LOOP;
END;
/

PL/SQL procedure successfully completed.

Elapsed: 00:00:07.878
SQL> CREATE OR REPLACE PACKAGE BODY dummy_hook AS
   PROCEDURE pre_ins IS
   BEGIN
      dbms_output.put('-');
   END pre_ins;
END dummy_hook;
/

Package body DUMMY_HOOK compiled

Elapsed: 00:00:00.029
SQL> -- with hook body
SQL> BEGIN
   FOR i IN 1..1E6 LOOP
      dummy_api.ins;
   END LOOP;
END;
/

PL/SQL procedure successfully completed.

Elapsed: 00:00:00.632

It makes sense to provide a body with a NULL implementation to avoid the small overhead of handling the missing body exception.

Nonetheless, the way the business logic is separated from the generated code is one of the many things I like about Bitemp Remodeler.

Download Bitemp Remodeler from the Download section on my blog or install it directly via the SQL Developer update site http://update.oddgen.org/

The post Bitemp Remodeler v0.1.0 Released appeared first on Philipp Salvisberg's Blog.

]]>