There are only two hard things in Computer Science: cache invalidation and naming things.
— Phil Karlton

IT is my daily life. And this quote is so true. Lately, I’ve been thinking much more than usual about naming, and that names really matter. This led to some refactoring activities.

Why Is Naming Important?

When a person with German mother tongue hears the word “eagle”, she or he automatically associates it with a “hedgehog”. Simply because the German word for it (“Igel”) is pronounced exactly the same. Of course, language skills and the concrete context play a role. The point is, a wrong association is likely. When we give a name to a thing, we basically want to avoid such false associations. In the best case, they are not helpful. In the worst case, this leads to rejection, as the next example shows.

In 1982 Mitsubishi Motors launched a SUV with the name “Pajero”. This name had to be changed in some regions, because “pajero” means “wanker” in Spanish. This example also shows that it is more important what others think about a name than we do.

In IT we have to name many things. Databases, schemas, tables, columns, views, packages, triggers, variables, fields, methods, classes, modules, components, products, etc. etc. Using an established name with a known and accepted definition help others to understand it better.

When we use a name, it is actually associated with a definition and properties, whether we like it or not. When names have a common and widely accepted meaning, it simplifies the communication. For example “banana”. Everybody knows what it means. Merriam-Webster’s definition is:

An elongated usually tapering tropical fruit with soft pulpy flesh enclosed in a soft usually yellow rind.

And I am sure that each of us could add a few characteristics to this definition.

Why Is Naming Difficult?

A name must fulfil many characteristics. For example

• Short
• Fitting (naturally relates to the intended meaning, and characteristics)
• Easy to spell, pronounce, remember
• Not associated with unwanted characteristics
• Common and widely accepted meaning and definition, that fits the intention (for names without commercial value)
• New, not used already (for marketable names)

Depending on context there are some goal conflicts. However, even without a major conflict, it is difficult to name something adequately in the early stages. Because we do not know enough about the thing we want to name. Hence, we use an iterative approach. We name something (e.g. an entity, package or class) and while working on it we find out that the name does not fit (anymore) and we change it. Maybe we split the thing and have to name now two things, etc. etc.

Finding a fitting name means doing some research. How have others named that thing? What is the definition of it? Does it fit 100 percent? This is an interesting and instructive work. In any case, it takes time. And at the time we need a new name, we want it now (e.g. when a wizard asks for a name). We can always rename it later, right? – Technically yes. And often we do. But the longer we wait, the less likely we are renaming.

Are Some Names More Important Than Others?

Yes. The more visible a name is the more important it is.

For example, the names behind an API are very easy to change. We do not have to ask anyone before changing it. It’s no problem as long as the API provides the same results. That’s one of the reasons we strive for tight APIs, right? To get some leeway.

As soon as others are involved, we are not fully in control of the change anymore. For example, when I change a name in one of my blog posts, this change is visible immediately to everyone visiting my blog. But I cannot control the caches of others, like search engines, blog mirrors and other services that copy web content to third-party storages. Remember, cache invalidation is the other hard thing in IT.

As a consequence, before we release an artefact that becomes visible to others, we should take some time to verify the used names. We cannot take back what we’ve said (at least not completely). However, we are in control of what we say in the future.

Banned Names on This Blog

Some terms (names) were discussed recently (again) due to a series of sad events. I used these terms as well. I never really thought about them as “bad”. However, I’ve changed my mind. I’m part of the problem. And I do not like it. One thing I can do is to stop using terms, that a large group of people associate with slavery and racism. No big deal, right?

This is another quote I like very much:

One cannot not communicate
— Paul Watzlawick

It is difficult to draw a line for certain terms. However, I believe that “you cannot not decide”. You decide either explicitly or implicitly. Of course, very seldom something is purely black or white. It’s much more often a shade of grey. Some decision take some time. And that’s okay. But it is impossible to postpone a decision forever. At a certain point, it becomes a decision.

So, I decided to decommission some terms on this blog and introduce new ones. Here’s the list:

Finding alternative names was surprisingly easy because others had already done the work and defined alternative names. They existed for years…

Master Data

However, finding an alternative for master data was harder. I reached out to my friends on Twitter. And got some helpful feedback. Finally, Robert Marti suggested having a look at Malcolm Chisholm‘s book Managing Reference Data in Enterprise Databases. On page 258ff the different data classes are defined and explained. The book is from 2000. In the meantime Malcolm Chisholm has published revised definitions here and here.

In the next subchapter, I repeat the definition of the data groups defined by Malcolm Chisholm on slide 5 in this deck. I like these definitions and plan to use them in the future.

Metadata

The data that describes all aspects of an enterprise’s information assets, and enables the enterprise to effectively use and manage these assets.

Here it is confined to the structure of databases. Found in a database’s system catalog. Sometimes included in database tables.

Reference Data

Any kind of data that is used solely to categorize other data found in a database, or solely for relating data in a database to information beyond the boundaries of the enterprise.

Codes and descriptions. Tables containing this data usually have just a few rows and columns.

Transaction Structure Data

Data that represents the direct participants in a transaction, and which must be present before a transaction fires.

The parties to the transactions of the enterprise. E.g. Customer, Product.

Enterprise Structure Data

Data that permits business activity to be reported and/or analyzed by business responsibility.

Typically, data that describes the structure of the enterprise. E.g. organizational or financial structure.

Transaction Activity Data

Data that represents the operations an enterprise carries out.

Traditional focus of IT – in many enterprises the only focus.

Transaction Audit Data

Data that tracks the life cycle of individual transactions.

Includes application logs, database logs, web server logs.

Summary

You use a name to simplify communication. A name is a proxy for a longer definition and meaning. If the meaning is badly received by others and especially by the target community, this does not simplify communication. Using a different name sounds like a simple solution. Why not, if changing a name is simple enough?

In this case, I only had to edit a few blog posts. I handled them like typos. This means that I did not add any update information. I also had to register new URL redirects. That was straightforward. However, changing the branch name in 26 GitHub repositories was a bit more work than anticipated, because I also had to change URLs in several related files. For certain GitHub pages, I had to keep a non-default master branch. I suppose that sooner or later GitHub will allow me to get rid of them as well. If I had to change more repositories, I would probably automate this task.

Most of the time I spent finding an alternative name for “master data”. In the end, I learned something new and found good names and definitions. That will help me in the future.

The post Names Matter appeared first on Philipp Salvisberg's Blog.

Semantics and Granularity of Periods

In Flashback Data Archive Oracle defines periods with a half-open interval. This means that a point in time x is part of a period if x >= the start of the period and x < the end of the period. It is no surprise that Oracle uses also half-open intervals for Temporal Validity. The following figure visualizes the principle:

The advantage of a half-open interval is that the end of a preceding period is identical to the start of the subsequent period. Thus there is no gap and the granularity of a period (year, month, day, second, millisecond, nanosecond, etc.) is irrelevant. The disadvantage is that querying data at a point in time using a traditional WHERE clause is a bit more verbose compared to closed intervals since BETWEEN conditions are not applicable.

Furthermore, Oracle uses NULL for -∞ und +∞. Considering this information the WHERE clause to filter the currently valid periods looks as follows:

WHERE (vt_start IS NULL OR vt_start <= SYSTIMESTAMP)
  AND (vt_end IS NULL OR vt_end > SYSTIMESTAMP)

Use of Temporal Periods

In an entity-relationship model temporal periods may be used for transaction structure data, enterprise structure data or reference data. For transaction activity data we do not need temporal periods since the data itself contains one or more timestamps. Corrections may be done through a reversal or difference posting logic, similar to bookkeeping transactions.

The situation is similar in a dimensional model. Dimensions correspond to transaction structure data, enterprise structure and reference data and may have a temporal period (e.g. slowly changing dimensions type 2). Facts do not have temporal periods. Instead, they are modeled with one or more relationships to the time dimension. A fact is immutable. Changes are applied through new facts using a reversal or difference posting logic.

Transaction Time – TT

A flight data recorder collects and records various metrics during a flight to allow the reconstruction of the past. The transaction or system time in a data model is comparable to the functionality of such a flight data recorder. A table with a transaction time axis allows to query the current and the past state, but changes in the past or the future are not possible.

Example: Scott becomes a manager. The change of the job description from “Analyst” to “Manager” was entered into the system on April 15, 2013, at 15:42:42. The previous description Analyst is terminated at this point in time and the new description Manager becomes current at exactly the same point in time.

Oracle supports the transaction time with Flashback Data Archive (formally known as Total Recall). Using Flashback Data Archive you may query a consistent state of the past.

SCN	Session A	Session B
1	INSERT INTO emp    (empno, ename, job, sal, deptno) VALUES    (4242, 'CARTER', 'CLERK', '2400', 20);
2	SELECT COUNT(*)   FROM emp; -- 15 rows
3		SELECT COUNT(*)   FROM emp; -- 14 rows
4	COMMIT;

Tab. 1: Consistent View of the Past

What is the result of the query “SELECT COUNT(*) FROM emp AS OF SCN 3” based on table 1 above? – 14 rows. This is a good and reasonable representation of the past. However, it also shows, that the consistent representation of the past is a matter of definition and in this case, it does not represent the situation of session A.

Valid Time – VT

The valid time describes the period during which something in the real world is considered valid. This period is independent of the entry into the system and therefore needs to be maintained explicitly. Changes and queries are supported in the past as well as in the future.

Example: Scott becomes a manager. The change of the job description from “Analyst” to “Manager” is valid from January 1 2014. The previous description Analyst is terminated at this point in time and the new description Manager becomes valid at exactly the same point in time. It is irrelevant when this change is entered into the System.

Decision Time – DT

The decision time describes the date and time a decision has been made. This point in time is independent of an entry into the System and is not directly related to the valid time period. Future changes are not possible.

Example: Scott becomes manager. The decision to change the job description from “Analyst” to “Manager” was made on March 24 2013. The previous job description Analyst is terminated on the decision time axis at this point in time and the new description Manager becomes current at exactly the same point in time on the decision time axis. It is irrelevant when this change is entered into the System and it is irrelevant when Scott may call himself officially a manager.

Historization Types

On one hand, the historization types are based on the time dimensions visualized in figure 2 and on the other hand categorized on the combination of these time dimensions. In this post only the most popular and generic time periods are covered. However, depending on the requirements additional, specific time periods are conceivable.

Non-temporal models do not have any time dimensions (e.g. EMP and DEPT in Schema SCOTT).

Uni-temporal models use just one time dimension (e.g. transaction time or valid time).

Bi-temporal models use exactly two time dimensions (e.g. transaction time and valid time).

Multi-temporal models use at least three time dimensions.

Tri-temporal  models are based exactly on three time dimensions.

Temporal Validity

The feature Temporal Validity covers the DDL and DML enhancements in Oracle 12c concerning temporal data management. The statements CREATE TABLE, ALTER TABLE and DROP TABLE have been extended by a new PERIOD FOR clause. Here is an example:

SQL> ALTER TABLE dept ADD (
  2     vt_start DATE,
  3     vt_end   DATE,
  4     PERIOD FOR vt (vt_start, vt_end)
  5  );

SQL> SELECT * FROM dept;	

    DEPTNO DNAME          LOC           VT_START   VT_END
---------- -------------- ------------- ---------- ----------
        10 ACCOUNTING     NEW YORK
        20 RESEARCH       DALLAS
        30 SALES          CHICAGO
        40 OPERATIONS     BOSTON

VT names the period and is a hidden column. The association of the VT period to the VT_START and VT_END columns is stored in the Oracle Data Dictionary in the table SYS_FBA_PERIOD. You need a dedicated ALTER TABLE call for every additional period.

For every period a  constraint is created to enforce positive time periods (VT_START < VT_END). However, it is not possible to define temporal constraints, e.g. prohibit overlapping periods, gaps, or orphaned parent/child periods.

Oracle 12c does not deliver support for temporal DML. Desirable would be for example:

• insert, update delete for a given period
• update a subset of columns for a given period
• merge of connected and identical periods

Hence temporal changes have to be implemented as a series of conventional DML. Here is an example:

SQL> UPDATE dept SET vt_end = DATE '2014-01-01' WHERE deptno = 30;

SQL> INSERT INTO dept (deptno, dname, loc, vt_start)
  2       VALUES (30, 'SALES', 'SAN FRANCISCO', DATE '2014-01-01');

SQL> SELECT * FROM dept WHERE deptno = 30 ORDER BY vt_start NULLS FIRST;

 DEPTNO    DNAME          LOC           VT_START   VT_END
---------- -------------- ------------- ---------- ----------
        30 SALES          CHICAGO                  2014-01-01
        30 SALES          SAN FRANCISCO 2014-01-01

Temporal Flashback Query

The feature Temporal Flashback Query covers query enhancements in Oracle 12c concerning temporal data. Oracle extended the existing Flashback Query interfaces. The FLASHBACK_QUERY_CLAUSE of the SELECT statement has been extended by a PERIOD FOR clause. Here is an example:

SQL> SELECT *
  2    FROM dept AS OF PERIOD FOR vt DATE '2015-01-01'
  3   ORDER BY deptno;

    DEPTNO DNAME          LOC           VT_START   VT_END
---------- -------------- ------------- ---------- ----------
        10 ACCOUNTING     NEW YORK
        20 RESEARCH       DALLAS
        30 SALES          SAN FRANCISCO 2014-01-01
        40 OPERATIONS     BOSTON

Instead of “AS OF PERIOD FOR” you may also use “VERSIONS PERIOD FOR”. However, it is important to notice that you may not define multiple PERIOD FOR clauses. Hence you need to filter additional temporal periods in the WHERE clause.

The PERIOD FOR clause is not applicable for views. For views, the enhancement in the PL/SQL package DBMS_FLASHBACK_ARCHIVE is interesting, especially the procedures ENABLE_AT_VALID_TIME and DISABLE_ASOF_VALID_TIME to manage a temporal context. Here is an example:

SQL> BEGIN
  2     dbms_flashback_archive.enable_at_valid_time(
  3        level      => 'ASOF', 
  4        query_time => DATE '2015-01-01'
  5     );
  6  END;
  7  /

SQL> SELECT * FROM dept ORDER BY deptno;

    DEPTNO DNAME          LOC           VT_START    VT_END
---------- -------------- ------------- ---------- ----------
        10 ACCOUNTING     NEW YORK
        20 RESEARCH       DALLAS
        30 SALES          SAN FRANCISCO 2014-01-01
        40 OPERATIONS     BOSTON

Currently, it is not possible to define a temporal period and therefore the context is applied to every temporal period. In these cases, you have to set the context via the WHERE clause.

A limitation of Oracle 12.1.0.1 is that Temporal Flashback Query predicates are not applied in multitenant configuration. The PERIOD FOR clause in the SELECT statement and the DBMS_FLASHBACK_ARCHIVE.ENABLE_AT_VALID_TIME calls are simply ignored. This limitation has been lifted with Oracle 12.1.0.2.

Another limitation is, that Oracle 12 does not provide support for temporal joins and temporal aggregations.

Tri-temporal Data Model

The following data model is based on the EMP/DEPT model in the schema SCOTT. The table EMPV implements three temporal dimensions:

• Transaction time (TT) with Flashback Data Archive
• Valid time (VT) with Temporal Validity
• Decision time (DT) with Temporal Validity

The table EMP is reduced to the primary key (EMPNO) which is not temporal. This allows to define and enable the foreign key constraint EMPV_EMP_MGR_FK.

The following six events will be represented with this model.

No	Transaction Time (TT)	Valid Time (VT)	Decision Time (DT)	Action
#1	1			Initial load from SCOTT.EMP table
#2	2	1990-01-01		Change name from SCOTT to Scott
#3	3	1991-04-01		Scott leaves the company
#4	4	1991-10-01		Scott rejoins
#5	5	1989-01-01		Change job from ANALYST TO Analyst
#6	6	2014-01-01	2013-03-24	Change job to Manager and double salary

Tab. 2: Events

After the processing of all 6 events the periods for employee 7788 (Scott) in the table EMPV may be queried as follows. The transaction time is represented as the System Change Number SCN.

SQL> SELECT dense_rank() OVER(ORDER BY versions_startscn) event_no, empno, ename, job,
  2         sal, versions_startscn tt_start, versions_endscn tt_end,
  3         to_char(vt_start,'YYYY-MM-DD') vt_start, to_char(vt_end,'YYYY-MM-DD') vt_end,
  4         to_CHAR(dt_start,'YYYY-MM-DD') dt_start, to_char(dt_end,'YYYY-MM-DD') dt_end
  5    FROM empv VERSIONS BETWEEN SCN MINVALUE AND MAXVALUE
  6   WHERE empno = 7788 AND versions_operation IN ('I','U')
  7   ORDER BY tt_start, vt_start NULLS FIRST, dt_start NULLS FIRST;

# EMPNO ENAME JOB       SAL TT_START   TT_END VT_START   VT_END     DT_START   DT_END
-- ----- ----- ------- ----- -------- -------- ---------- ---------- ---------- ----------
 1  7788 SCOTT ANALYST  3000  2366310  2366356
 2  7788 SCOTT ANALYST  3000  2366356  2366559            1990-01-01
 2  7788 Scott ANALYST  3000  2366356  2366408 1990-01-01
 3  7788 Scott ANALYST  3000  2366408  2366559 1990-01-01 1991-04-01
 4  7788 Scott ANALYST  3000  2366424  2366559 1991-10-01
 5  7788 SCOTT ANALYST  3000  2366559                     1989-01-01
 5  7788 SCOTT Analyst  3000  2366559          1989-01-01 1990-01-01
 5  7788 Scott Analyst  3000  2366559          1990-01-01 1991-04-01
 5  7788 Scott Analyst  3000  2366559  2366670 1991-10-01
 6  7788 Scott Analyst  3000  2366670          1991-10-01                       2013-03-24
 6  7788 Scott Analyst  3000  2366670          1991-10-01 2014-01-01 2013-03-24
 6  7788 Scott Manager  6000  2366670          2014-01-01            2013-03-24

7 rows have been changed or added based on event #5 at the transaction time 2366559. It clearly shows that DML operations in a temporal model are not trivial. All the more support in that area for VT and DT is missed.

The next query filters the data for Scott on the transaction time (SYSDATE=default), valid time (2014-01-01) and decision time (2013-04-01). This way the result is reduced exactly to a single row.

SQL> SELECT empno, ename, job, sal,
  2         to_char(vt_start,'YYYY-MM-DD') AS vt_start,
  3         to_char(vt_end,'YYYY-MM-DD') AS vt_end,
  4         to_CHAR(dt_start,'YYYY-MM-DD') AS dt_start,
  5         to_char(dt_end,'YYYY-MM-DD') AS dt_end
  6    FROM empv AS OF period FOR dt DATE '2013-04-01'
  7   WHERE empno = 7788 AND
  8         (vt_start <= DATE '2014-01-01' OR vt_start IS NULL) AND
  9         (vt_end > DATE '2014-01-01' OR vt_end IS NULL)
 10   ORDER BY vt_start NULLS FIRST, dt_start NULLS FIRST;

EMPNO ENAME JOB       SAL VT_START   VT_END     DT_START   DT_END
----- ----- ------- ----- ---------- ---------- ---------- ----------
 7788 Scott Manager  6000 2014-01-01            2013-03-24

Queries on multi-temporal data are relatively simple if all time periods are filtered at a point in time. The AS OF PERIOD clause (for DT) simplifies the query, but the complexity of a traditional WHERE condition (for VT) is not much higher.

Conclusion

The support for temporal data management in Oracle 12c is based on sound concepts, but the implementation is currently incomplete. I miss mainly a temporal DML API, temporal integrity constraints, temporal joins and temporal aggregations. I recommend using Oracle’s semantics for periods (half-open intervals, NULL for +/- infinity) in existing models, to simplify the migration to Temporal Validity.

In the real world, we use a lot of temporal dimensions, consciously or unconsciously at the same time. However, in data models, every additional temporal dimension increases the complexity significantly. Data models are simplifications of the real world, based on requirements and a limited budget. I do not recommend using bi-temporality or even multi-temporality as a universal design pattern. Quite the contrary I recommend determining and documenting the reason for a temporal dimension per entity to ensure that temporal dimensions are used consciously and not modeled unnecessarily.

Oracle’s Flashback Data Archive is good, transparent and since Oracle 11.2.0.4 also a cost-free option to implement requirements regarding the transaction time. For all other time dimensions such as the valid time and the decision time I recommend using standardized tooling to apply DML to temporal data.

Last update on 2015-10-24, amendments to match limitations of Oracle version 12.1.0.2.4.

The post Multi-temporal Features in Oracle 12c appeared first on Philipp Salvisberg's Blog.