I have used the Oracle Database for many years. And I use hints. For experiments, but also in production code. There are cases when you know more than the Oracle Database. For example about the cardinality of a data source or the number of result rows to process or the number of expected executions of a statement. Hints are a way to provide additional information, limit the solution space and enable the database to do a better job. That’s a good thing.

Hints Are Instructions

Hints are passed as special comments at a certain position in SQL statements. They are comments, but they are also instructions. They have to be followed. However, there are cases when hints are not applicable. For example when you request the optimizer to use an index when there is no index defined for the underlying table. In such a case the Oracle Database has basically two options. Either throw an error or ignore the invalid instruction and find another solution. The Oracle Database does the latter.

Hint Report

Starting with version 19c you can produce a hint report that reveals unused hints. Here’s an example:

create table t (c1 integer, c2 varchar2(20));
insert into t values (1, 'one');
insert into t values (2, 'two');
select /*+ index(t) */ * from t where c1 > 0;
select * from dbms_xplan.display_cursor(format => 'basic +hint_report');

EXPLAINED SQL STATEMENT:
------------------------
select /*+ index(t) */ * from t where c1 > 0
 
Plan hash value: 1601196873
 
------------------------------------------
| Id  | Operation                 | Name |
------------------------------------------
|   0 | SELECT STATEMENT          |      |
|   1 |  TABLE ACCESS STORAGE FULL| T    |
------------------------------------------
 
Hint Report (identified by operation id / Query Block Name / Object Alias):
Total hints for statement: 1 (U - Unused (1))
---------------------------------------------------------------------------
 
   1 -  SEL$1 / "T"@"SEL$1"
         U -  index(t)

The hint index(t) defined on line 4 is valid, but it’s reported as unused on line 25. No wonder. There is no index defined on table t.

Let’s create an index and rerun the query.

create unique index t_c1_i on t(c1);
select /*+ index(t) */ * from t where c1 > 0;
select * from dbms_xplan.display_cursor(format => 'basic +hint_report');

EXPLAINED SQL STATEMENT:
------------------------
select /*+ index(t) */ * from t where c1 > 0
 
Plan hash value: 2704710798
 
------------------------------------------------------
| Id  | Operation                           | Name   |
------------------------------------------------------
|   0 | SELECT STATEMENT                    |        |
|   1 |  TABLE ACCESS BY INDEX ROWID BATCHED| T      |
|   2 |   INDEX RANGE SCAN                  | T_C1_I |
------------------------------------------------------
 
Hint Report (identified by operation id / Query Block Name / Object Alias):
Total hints for statement: 1
---------------------------------------------------------------------------
 
   1 -  SEL$1 / "T"@"SEL$1"
           -  index(t)

Now the hint index(t) defined on line 2 is reported as used on line 24.

Mixing Hints and Comments

What happens if we mix hints and comments? It depends on where you place the comment. Let’s look at the next example.

select /*+ index(t) forcing unnecessary index access */ * from t where c1 > 0;
select * from dbms_xplan.display_cursor(format => 'basic +hint_report');

EXPLAINED SQL STATEMENT:
------------------------
select /*+ index(t) forcing unnecessary index access */ * from t where 
c1 > 0
 
Plan hash value: 2704710798
 
------------------------------------------------------
| Id  | Operation                           | Name   |
------------------------------------------------------
|   0 | SELECT STATEMENT                    |        |
|   1 |  TABLE ACCESS BY INDEX ROWID BATCHED| T      |
|   2 |   INDEX RANGE SCAN                  | T_C1_I |
------------------------------------------------------
 
Hint Report (identified by operation id / Query Block Name / Object Alias):
Total hints for statement: 4 (E - Syntax error (3))
---------------------------------------------------------------------------
 
   1 -  SEL$1
         E -  forcing
         E -  index 
         E -  unnecessary
 
   1 -  SEL$1 / "T"@"SEL$1"
           -  index(t)

The comment forcing unnecessary index access on line 1 is interpreted as a series of hints and reported as errors on lines 24 to 26. The token access was not reported. However, the hint index(t) was reported as used on line 29.

What happens if we move the comment to the beginning?

select /*+ forcing unnecessary index access index(t) */ * from t where c1 > 0;
select * from dbms_xplan.display_cursor(format => 'basic +hint_report');

EXPLAINED SQL STATEMENT:
------------------------
select /*+ forcing unnecessary index access index(t) */ * from t where 
c1 > 0
 
Plan hash value: 2704710798
 
------------------------------------------------------
| Id  | Operation                           | Name   |
------------------------------------------------------
|   0 | SELECT STATEMENT                    |        |
|   1 |  TABLE ACCESS BY INDEX ROWID BATCHED| T      |
|   2 |   INDEX RANGE SCAN                  | T_C1_I |
------------------------------------------------------
 
Hint Report (identified by operation id / Query Block Name / Object Alias):
Total hints for statement: 3 (E - Syntax error (3))
---------------------------------------------------------------------------
 
   1 -  SEL$1
         E -  forcing
         E -  index 
         E -  unnecessary

The same invalid hints are reported as before on lines 24 to 26. However, the hint index(t) was used but not reported as such. This seems to be a limitation of the current hint report in the Oracle Database 21c.

Anyways, it clearly shows that you should not mix comments and hints. Instead, you should write it like this:

select /* forcing unnecessary index access */ /*+ index(t) */ * from t where c1 > 0;
select * from dbms_xplan.display_cursor(format => 'basic +hint_report');

EXPLAINED SQL STATEMENT:
------------------------
select /* forcing unnecessary index access */ /*+ index(t) */ * from t 
where c1 > 0
 
Plan hash value: 2704710798
 
------------------------------------------------------
| Id  | Operation                           | Name   |
------------------------------------------------------
|   0 | SELECT STATEMENT                    |        |
|   1 |  TABLE ACCESS BY INDEX ROWID BATCHED| T      |
|   2 |   INDEX RANGE SCAN                  | T_C1_I |
------------------------------------------------------
 
Hint Report (identified by operation id / Query Block Name / Object Alias):
Total hints for statement: 1
---------------------------------------------------------------------------
 
   1 -  SEL$1 / "T"@"SEL$1"
           -  index(t)

Now the hint index(t) is reported as used. All good, right?

The Problem

I like statically-type languages. Mainly because errors are reported at compile time whenever possible. However, to check hints I need to produce an explain plan. This is possible for a single statement only. This is cumbersome especially when you write code in PL/SQL. As far as I know, there is no option to produce a compile error for invalid hints.

I recently reviewed a system and found a lot of invalid hints. Here are some real-life hints copied from a production code base:

• /*+ parallel 4 */
• /*+ no_xml_query_rewrite +materialize */
• /*+ materialized */
• /*+ first rows cardinality (a,10) */
• /*+ append nologging */
• /*+ le ading(g) u se_nl(g) u se_hash(p, b) */

The last example is a kind of commented-out hint series. In this case, it’s clearly commented-out code. But if you see just a single hint like /*+ le ading(g) */ in the code, you do not know if the space after le was entered intentionally or by accident.

So, how can we identify invalid hints in our code?

Step 1 – Distinguish Between Comments and Hints

We can configure Oracle’s SQL Developer to show hints in a different colour than comments. Here’s the screenshot of an example I showed above:

Go to this GitHub repository and follow the instructions to configure your SQL Developer installation accordingly. See also this blog post for more information about the Arbori code that makes such code highlighting possible.

This step make hints stand out in your code. However, it does not reveal invalid hints.

Step 2 – Install db* CODECOP for SQL Developer

To reveal invalid hints we need a linter. A tool that does some static code analysis. db* CODECOP is such a tool suite. The SQL Developer extension is available for free. It checks the editor content for violations of the Trivadis PL/SQL & SQL Coding Guidelines. Furthermore, db* CODECOP allows you to implement custom guideline checks. The example GitHub repository provides the following four guideline checks regarding hints:

• G-9600: Never define more than one comment with hints.
• G-9601: Never use unknown hints.
• G-9602: Always use the alias name instead of the table name.
• G-9603: Never reference an unknown table/alias.

To install db* CODECOP and these additional custom guideline checks follow the instructions in this GitHub repository.

Finding Wrong Hints With db* CODECOP

I asked my followers on Twitter if this hint is valid:

The result is not really representative. However, 25% thought that /*+ +materialize */ is a valid hint.

Checking the code with db* CODECOP reveals that the hint is invalid and the majority of the poll participants were right.

Verify Result

But is the result of db* CODECOP correct? The following explain plan shows that the hint /*+ +materialize */ is not reported at all. It’s treated as a comment. Another example is where the hint report is incomplete.

with e as (
   select /*+ +materialize */ *
     from emp
    where deptno = 10
)
select *
  from e;
select * from dbms_xplan.display_cursor(format => 'basic +hint_report');

EXPLAINED SQL STATEMENT:
------------------------
with e as (    select /*+ +materialize */ *      from emp     where 
deptno = 10 ) select *   from e
 
Plan hash value: 3956160932
 
------------------------------------------
| Id  | Operation                 | Name |
------------------------------------------
|   0 | SELECT STATEMENT          |      |
|   1 |  TABLE ACCESS STORAGE FULL| EMP  |
------------------------------------------

Let’s run the same query after removing the extra + in the hint:

with e as (
   select /*+ materialize */ *
     from emp
    where deptno = 10
)
select *
  from e;
select * from dbms_xplan.display_cursor(format => 'basic +hint_report');

EXPLAINED SQL STATEMENT:
------------------------
with e as (    select /*+ materialize */ *      from emp     where 
deptno = 10 ) select *   from e
 
Plan hash value: 3494145522
 
--------------------------------------------------------------------------------
| Id  | Operation                                | Name                        |
--------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                         |                             |
|   1 |  TEMP TABLE TRANSFORMATION               |                             |
|   2 |   LOAD AS SELECT (CURSOR DURATION MEMORY)| SYS_TEMP_DFD9DB186_8AAEBD74 |
|   3 |    TABLE ACCESS STORAGE FULL             | EMP                         |
|   4 |   VIEW                                   |                             |
|   5 |    TABLE ACCESS STORAGE FULL             | SYS_TEMP_DFD9DB186_8AAEBD74 |
--------------------------------------------------------------------------------
 
Hint Report (identified by operation id / Query Block Name / Object Alias):
Total hints for statement: 1
---------------------------------------------------------------------------
 
   2 -  SEL$1
           -  materialize

Now, the materialize hint has an effect on the execution plan and the hint is reported as used on line 33.

Conclusion

I believe that hints are required for certain use cases. You may have a different opinion. However, if you are using hints in your code you should ensure that they are valid. db* CODECOP can help you to do that. The SQL Developer extension is free. Just use it.

The post Finding Wrong Hints appeared first on Philipp Salvisberg's Blog.

There are only two hard things in Computer Science: cache invalidation and naming things.
— Phil Karlton

IT is my daily life. And this quote is so true. Lately, I’ve been thinking much more than usual about naming, and that names really matter. This led to some refactoring activities.

Why Is Naming Important?

When a person with German mother tongue hears the word “eagle”, she or he automatically associates it with a “hedgehog”. Simply because the German word for it (“Igel”) is pronounced exactly the same. Of course, language skills and the concrete context play a role. The point is, a wrong association is likely. When we give a name to a thing, we basically want to avoid such false associations. In the best case, they are not helpful. In the worst case, this leads to rejection, as the next example shows.

In 1982 Mitsubishi Motors launched a SUV with the name “Pajero”. This name had to be changed in some regions, because “pajero” means “wanker” in Spanish. This example also shows that it is more important what others think about a name than we do.

In IT we have to name many things. Databases, schemas, tables, columns, views, packages, triggers, variables, fields, methods, classes, modules, components, products, etc. etc. Using an established name with a known and accepted definition help others to understand it better.

When we use a name, it is actually associated with a definition and properties, whether we like it or not. When names have a common and widely accepted meaning, it simplifies the communication. For example “banana”. Everybody knows what it means. Merriam-Webster’s definition is:

An elongated usually tapering tropical fruit with soft pulpy flesh enclosed in a soft usually yellow rind.

And I am sure that each of us could add a few characteristics to this definition.

Why Is Naming Difficult?

A name must fulfil many characteristics. For example

• Short
• Fitting (naturally relates to the intended meaning, and characteristics)
• Easy to spell, pronounce, remember
• Not associated with unwanted characteristics
• Common and widely accepted meaning and definition, that fits the intention (for names without commercial value)
• New, not used already (for marketable names)

Depending on context there are some goal conflicts. However, even without a major conflict, it is difficult to name something adequately in the early stages. Because we do not know enough about the thing we want to name. Hence, we use an iterative approach. We name something (e.g. an entity, package or class) and while working on it we find out that the name does not fit (anymore) and we change it. Maybe we split the thing and have to name now two things, etc. etc.

Finding a fitting name means doing some research. How have others named that thing? What is the definition of it? Does it fit 100 percent? This is an interesting and instructive work. In any case, it takes time. And at the time we need a new name, we want it now (e.g. when a wizard asks for a name). We can always rename it later, right? – Technically yes. And often we do. But the longer we wait, the less likely we are renaming.

Are Some Names More Important Than Others?

Yes. The more visible a name is the more important it is.

For example, the names behind an API are very easy to change. We do not have to ask anyone before changing it. It’s no problem as long as the API provides the same results. That’s one of the reasons we strive for tight APIs, right? To get some leeway.

As soon as others are involved, we are not fully in control of the change anymore. For example, when I change a name in one of my blog posts, this change is visible immediately to everyone visiting my blog. But I cannot control the caches of others, like search engines, blog mirrors and other services that copy web content to third-party storages. Remember, cache invalidation is the other hard thing in IT.

As a consequence, before we release an artefact that becomes visible to others, we should take some time to verify the used names. We cannot take back what we’ve said (at least not completely). However, we are in control of what we say in the future.

Banned Names on This Blog

Some terms (names) were discussed recently (again) due to a series of sad events. I used these terms as well. I never really thought about them as “bad”. However, I’ve changed my mind. I’m part of the problem. And I do not like it. One thing I can do is to stop using terms, that a large group of people associate with slavery and racism. No big deal, right?

This is another quote I like very much:

One cannot not communicate
— Paul Watzlawick

It is difficult to draw a line for certain terms. However, I believe that “you cannot not decide”. You decide either explicitly or implicitly. Of course, very seldom something is purely black or white. It’s much more often a shade of grey. Some decision take some time. And that’s okay. But it is impossible to postpone a decision forever. At a certain point, it becomes a decision.

So, I decided to decommission some terms on this blog and introduce new ones. Here’s the list:

Finding alternative names was surprisingly easy because others had already done the work and defined alternative names. They existed for years…

Master Data

However, finding an alternative for master data was harder. I reached out to my friends on Twitter. And got some helpful feedback. Finally, Robert Marti suggested having a look at Malcolm Chisholm‘s book Managing Reference Data in Enterprise Databases. On page 258ff the different data classes are defined and explained. The book is from 2000. In the meantime Malcolm Chisholm has published revised definitions here and here.

In the next subchapter, I repeat the definition of the data groups defined by Malcolm Chisholm on slide 5 in this deck. I like these definitions and plan to use them in the future.

Metadata

The data that describes all aspects of an enterprise’s information assets, and enables the enterprise to effectively use and manage these assets.

Here it is confined to the structure of databases. Found in a database’s system catalog. Sometimes included in database tables.

Reference Data

Any kind of data that is used solely to categorize other data found in a database, or solely for relating data in a database to information beyond the boundaries of the enterprise.

Codes and descriptions. Tables containing this data usually have just a few rows and columns.

Transaction Structure Data

Data that represents the direct participants in a transaction, and which must be present before a transaction fires.

The parties to the transactions of the enterprise. E.g. Customer, Product.

Enterprise Structure Data

Data that permits business activity to be reported and/or analyzed by business responsibility.

Typically, data that describes the structure of the enterprise. E.g. organizational or financial structure.

Transaction Activity Data

Data that represents the operations an enterprise carries out.

Traditional focus of IT – in many enterprises the only focus.

Transaction Audit Data

Data that tracks the life cycle of individual transactions.

Includes application logs, database logs, web server logs.

Summary

You use a name to simplify communication. A name is a proxy for a longer definition and meaning. If the meaning is badly received by others and especially by the target community, this does not simplify communication. Using a different name sounds like a simple solution. Why not, if changing a name is simple enough?

In this case, I only had to edit a few blog posts. I handled them like typos. This means that I did not add any update information. I also had to register new URL redirects. That was straightforward. However, changing the branch name in 26 GitHub repositories was a bit more work than anticipated, because I also had to change URLs in several related files. For certain GitHub pages, I had to keep a non-default master branch. I suppose that sooner or later GitHub will allow me to get rid of them as well. If I had to change more repositories, I would probably automate this task.

Most of the time I spent finding an alternative name for “master data”. In the end, I learned something new and found good names and definitions. That will help me in the future.

The post Names Matter appeared first on Philipp Salvisberg's Blog.

Moved

I moved all product information including change logs and frequently asked questions to the following GitHub repositories:

• PL/SQL Analyzer
• PL/SQL Cop Command Line
• PL/SQL Cop Validators
• PL/SQL Cop for SonarQube
• PL/SQL Cop for SQL Developer
• PL/SQL Unwrapper for SQL Developer
• plscope-utils for SQL Developer

These repositories contain the product information and the software releases including the release history. The idea is to manage all issues in GitHub repositories, regardless of the public availability of the product source code. I’m sure this will simplify the work of all involved parties.

Stashed

I removed the forum from the main menu. However, it is still there. If you know the URL (e.g. by guessing or because you have some forum e-mails) then you may access it. For the time being, I keep it in read-only mode. However, I plan to delete the forum without migrating the content.

Kept

The Download area is still there. However, all links including download links point to other websites.

Summary

The move to GitHub is complete. I registered a lot of redirects. So I expect all links to salvis.com to work still and show the expected content. Please leave a comment, If you experience dead links. Thank you.

The post Moving to GitHub appeared first on Philipp Salvisberg's Blog.