How to Tune SQL Statement with LCASE function on index field?

Some business requirements may need to compare the lower case of an indexed column to a given string as a data retrieval criterion.

Here is an example SQL that retrieves records from the EMPLOYEE table employee if the lower case of the name is equal to the string ‘richard’.

select  *
  from employee
where LCASE(emp_name)=‘richard’

Here the following are the query plans of this SQL, it takes 17 seconds to finish. The query shows a “Full Table Scan Employee”  

You can see that this SQL cannot utilize index scan even if the emp_name is an indexed field. Let me add a “Force Index(emp_name_inx)“hint to the SQL and hope it can help MySQL SQL optimizer to use index scan, but it fails to enable the index scan anyway, so I add one more dummy condition “emp_name >= ””, it is an always true condition that emp_name should be greater or equal to a smallest empty character, it is used to increase the cost of not using emp_name_inx index. There is another condition added “emp_name is null” to correct this condition if emp_name is a null value.

select  *
from   employee force index(EMPS_NAME_INX)
where  LCASE(emp_name) = ‘richard’
     and ( emp_name >=
        or emp_name is null )

Here is the query plan of the rewritten SQL and it is running much faster. The new query plan shows that an Index Scan is used now and takes 2.79 seconds only.

This kind of rewrite can be achieved by Tosska SQL Tuning Expert for MySQL automatically, it shows that the rewrite is more than 6 times faster than the original SQL.

https://tosska.com/tosska-sql-tuning-expert-tse-for-mysql-2/

How to use ROWID to improve an UPDATE statement for Oracle?

Here the following is an Update SQL with a subquery that updates the EMPLOYEE table if the emp_dept satisfies the records returned from a subquery.

update  employee
   set  emp_name = ‘testing’
 where  emp_dept IN (select dpt_id
            from department
          where dpt_name like ‘A%’)
and emp_grade>2000

You can see Oracle uses a Hash join of the DEPARTMENT table and EMPLOYEE table to execute the update process. This query plan takes 1.96 seconds to complete and no index is used even though emp_dept, dpt_id, and emp_grade are indexed columns. It looks like the most expansive operation is the Table Access Full scan of the EMPLOYEE table.

Let’s rewrite the SQL into the following syntax to eliminate EMPLOYEE’s Table Access Full operation from the query plan.  The new subquery with the italic Bold text is used to force the EMPLOYEE to extract records with emp_dept in the DEPARTMENT table with the dpt_name like ‘A%’. The ROWID returned from the EMPLOYEE(subquery) is to make sure a more efficient table ROWID access to the outer EMPLOYEE table.

UPDATE  employee
SET   emp_name=‘testing’
WHERE   ROWID IN (SELECT  ROWID
          FROM   employee
          WHERE  emp_dept IN (SELECT  dpt_id
                      FROM   department
                      WHERE  dpt_name LIKE‘A%’))
     AND emp_grade > 2000

You can see the final query plan with this syntax has a better cost without full table access to the EMPLOYEE table. The new syntax takes 0.9 seconds and it is more than 2 times faster than the original syntax.

This kind of rewrite can be achieved by Tosska SQL Tuning Expert Pro for Oracle automatically, there is another SQL rewrite with similar performance, but it is not suitable to discuss in this short article, maybe I can discuss it later in my blog.

https://tosska.com/tosska-sql-tuning-expert-pro-tse-pro-for-oracle/

How to build indexes for multiple Max() functions for SQL Server?

For some SQL statements with multiple Max() functions in the select list and nothing in the Where clause, we have different methods to create new indexes to improve the SQL speed.

Here is an example SQL, it is to retrieve the maximum name and age from the employee table.
select   max(emp_name),
     max(emp_age)
from  employee

The following is the query plan that takes 9.27 seconds.

The SQL cannot be tuned by SQL syntax rewrite or hints injection, and the SSMS cannot recommend any index to improve the SQL.

For this kind of SQL that we can consider building a composite index or two individual indexes for emp_name and emp_age. A new composite of these two columns (emp_age, emp_name) can improve the SQL around 7 times. The following is the query plan shows that the new composite index is used, but it has to scan the entire index for these two stream aggregate operations before getting the max(emp_name) and max(emp_age).

How about if we build two individual indexes for emp_name and emp_age. The following is the result and query plan of these two indexes created. A Top operator selects the first row from each index and returns to the Stream Aggregate operation, and then a Nested Loops join the two maximum results together. It is 356 times much faster than the original SQL.

This kind of indexes recommendation can be achieved by Tosska SQL Tuning Expert Pro for SQL Server automatically:
Tosska SQL Tuning Expert Pro (TSES Pro™) for SQL Server – Tosska Technologies Limited

Measures to Improve Performance of SQL Query?

oracle database performance tuning

That’s not bad enough that you call it SQL whereas your boss pronounces it ‘sequel’. But also, you now suffer from “Super Slow Query Syndrome,” and sometimes, your questions bomb without effect.

Don’t worry. We have your back. We recently had a powwow with a lot of caffeine to think about our favorite tips to fix queries. With the help of this article, we will dig into how we can resolve SQL queries and improve the performance of SQL queries with new tips and tricks, such as action plans, references, wild cards, and much more.

In fact, we have combined all our famous skills into one, so you can increase your SQL intelligence by six minutes flat.

The issues faced by the companies in SQL Server performance often lead to focusing on using tuning tools and development strategies. This will help to analyze and process queries faster and eliminate operational issues, troubleshoot poor performance, avoid any chaos, or reduce the impact on the SQL Server database.

What is SQL Query Optimization?

Optimizing SQL Query is the process of writing considerate SQL queries to improve database performance. During development, the amount of data accessed and tested is not much. Thus, it becomes easy for developers to get a prompt response to their raised questions. But the problem starts when the project becomes live and large data starts to flood the database. These kinds of situations reduce the resolving process and performance.

A request for data or information from a database is called a Query, and you need to write a pre-defined set of code that is understandable to the database. Structured Query Language (SQL) and other query languages recover or manage data from related databases.

There are different formats to write a query in the database, using various algorithms. A query that is incomplete or written poorly can lead to a lot of resource consumption, and also can take a lot of time in execution, which possibly causes a loss in services. A proper query can reduce implementation time and lead to better SQL results.

SQL query optimization’s main purpose is to reduce response time and improve query performance, Reduce CPU performance time for faster results and reduce the number of resources used to improve the output.

Ways to Improve SQL Query Performance

Avoiding unnecessary columns in the SELECT section

To improve MySQL functionality, it’s recommended to specify columns in the SELECT section, instead of using SELECT*. As irrelevant columns create more load in the database, it slows down the performance of the whole system.

Using internal joining, rather than external joining if possible

Use external joining only if necessary. Excessive use of it not only limits database performance but also limits MySQL query options, resulting in slower SQL statements.

Using DISTINCT and UNION only if necessary

By using UNION and DISTINCT operators while there are no major objective results in unwanted filtering and reduced SQL performance. To improve the performance and bring efficiency to the process we can always use UNION ALL, rather than UNION.

Using the ORDER BY clause

To get more clear results it is important to use the ORDER BY clause. It not only brings 

advantages for database admins but also increases performance in its execution.

SQL Query Performance Tuning: Best Practice

SQL Query tuning is one of the fastest ways to improve the performance of SQL Server. Set procedures and processes are used to improve the performance and resolve the database-related queries this is called Tuning the SQL server. SQL tuning includes several features, including identifying which queries are slower and utilizing them to work more efficiently. Multiple communication databases like MySQL and SQL Server will benefit from SQL tuning.

The Database Performance Analyzer can attempt to troubleshoot server performance issues in the system. But these measures are expensive, and they may not work to solve the problem of slow-moving queries. Tuning SQL functionality helps you to identify poorly written SQL queries and invalid indexing conditions. After doing so, you may find that you do not need to invest in hardware upgrades or technical details.

Tuning SQL functionality can be difficult, especially if done manually. Believe it or not, the slight changes can have major effects on SQL Server and database performance. Hence, there is a need for practical SQL Query performance tools.

To conclude, generally, the best practices of SQL Query performance Tuning include proper indexing that can be done by the Execution Plan tool in SQL Server. Additionally, avoiding coding loops and correlating SQL subqueries.

Tosska SQL Tuning Expert (TSE™) v4 is one of the best SQL tuning tool available in the market. It helps in tuning the SQL even without any source code.

A Quick Guide to Stored Procedures in Oracle Database and SQL

sql tuning for MySQL

Stored procedures are increasing in popularity in Oracle Database and SQL Server because of quicker execution. Earlier, application code mostly resided in external programs. However, its shift toward database engine interiors compels database professionals to keep their memory requirements in mind.

This is as necessary as planning for times when the code related to database access will be present within the database. They also need to know how they can handle these stored procedures to maintain ideal database performance. We will look at some of these methods and the advantages of using stored procedures and triggers in the Oracle database.

Perks of Stored Procedures for Oracle Database Performance Tuning

Until recently, a majority of Oracle databases had limited code within their stored procedures. This shift in trends is because of the various advantages that come with placing larger amounts of code, such as the following:

Performance Improvement – Using more stored procedures means you don’t require Oracle database performance tuning as much. That’s because each of these only has to load once into the shared pool. Executing them, therefore, is quicker than running external code.

Code Segregation – The stored procedures have all the SQL codes which turn all the application programs into calls for those procedures. This is an improvement in the data retrieval process because changing databases gets simpler.

Therefore, one advantage you get through stored procedures is the ability to transfer large amounts of SQL code to the data dictionary. Doing this will enable you to perform SQL tuning without involving the application layer.

Group Data Easily – You can gather relational tables with data that shares certain behaviour before looking for Oracle performance tuning tips. Simply use Oracle stored procedures as methods, along with suitable naming conventions. For example, link the behaviour of the table data to the table name in the form of prefixes.

The users may then request the data dictionary to display all the traits connected to one table. This makes it more convenient to recognise and reuse code with the help of stored procedures.

Other Reasons Behind the Increasing Popularity of Stored Procedures

There are plenty of other reasons ‌stored procedures and triggers take less time in comparison with conventional code. One of these has something to do with SGA caching in Oracle database and SQL.

Once the shared pool within the SGA gets a hold of a stored procedure, it keeps it there until the procedure gets paged out from the memory. The SGA mostly does this to create space for other stored procedures. The paging out process takes place based on a Least Recently Used or LRU algorithm.

Two parameters help determine the amount of space that Oracle uses on startup. These are the Cache Size and the Shared Pool Size parameters. They also help users check how much storage space is available for various tasks. These include caching SQL code, data blocks, and stored procedures.

Stored procedures will run extremely fast once you load them into the shared pool’s RAM – as long as you can avoid pool thrashing. This is important because several procedures compete for varying quantities of memory in the shared pool. 

How to build indexes for slow first execution SQL – SQL Server?

You may suffer from SQL statements with a slow first execution time due to the long data cache process. The following SQL is simple that retrieves records from the EMPLOYEE table that if EMP_SALARY < 500000 and the result set is ordered by EMP_NAME.

Select emp_id,
    emp_name,
    emp_salary,
    emp_address,
    emp_telephone
from    employee
where  emp_salary < 500000
order by emp_name;

The following is the query plan that takes 9.51 seconds for the first execution and takes 0.99 seconds for the second execution without data cache.

The SQL cannot be tuned by SQL syntax rewrite or hints injection for both the first execution and the second execution, it is because SQL Server has selected the best query plan for this simple SQL statement. But the problem is that if the condition “where emp_salary < 500000” is changed; say from 500000 to 510000 or the EMPLOYEE data is flushed out from the memory, the execution time will then be prolonged up to 9.51 seconds.

Let’s see if we can build indexes to improve this situation. There is a common perception that a good index can help to improve both the first execution time and the second execution time. So, I use a tool to explore a lot of indexes configurations, but none of them can improve both executions’ performance. Here the following is the performance of the second execution with data cached for different indexes proposed by the tool. You can see the performance of “Index Set 1” is close to the original SQL performance with a little performance variation due to the system’s loading status and all other indexes sets are worse than the original SQL. Normally, we will give up the tuning of the SQL statement without even trying to see whether those recommended indexes are good for the first execution time.

I did a test for those recommended indexes to see whether they are helpful to improve the first execution time, it surprises me that the “Index Set 1” is tested with a significant improvement and improves the first execution time from 9.51 seconds to 0.65 seconds. It is a 14 times improvement that can make my database run more efficiently. So, you should be very careful to tune your SQL with new indexes that may not be good for your second execution with all data cached, but it may be very good for your first execution without data cached.

This kind of indexes recommendation can be achieved by Tosska SQL Tuning Expert Pro for SQL Server automatically.

Tosska SQL Tuning Expert Pro (TSES Pro™) for SQL Server – Tosska Technologies Limited

How to index SQL with aggregate function SQL for Oracle?

Here the following is an example SQL shows you that select the maximum emp_address which is not indexed in the EMPLOYEE table with 3 million records, the emp_grade is an indexed column.

select max(emp_address) from employee a
where emp_grade<4000

As 80% of the EMPLOYEE table’s records will be retrieved to examine the maximum emp_address string. The query plan of this SQL shows a Table Access Full on EMPLOYEE table is reasonable.

How many ways to build an index to improve this SQL?
Although it is simple SQL, there are still 3 ways to build an index to improve this SQL, the following are the possible indexes that can be built for the SQL, the first one is a single column index and the 2 and 3 are the composite index with a different order.
1. EMP_ADDRESS
2. EMP_GRADE, EMP_ADDRESS
3. EMP_ADDRESS, EMP_GRADE

Most people may use the EMP_ADDRESS as the first choice to improve this SQL, let’s see what the query plan is if we build a virtual index for the EMP_ADDRESS column in the following, you can see the estimated cost is reduced by almost half, but this query plan is finally not being used after the physical index is built for benchmarking due to actual statistics is collected.

The following query shows the EMP_ADDRESS index is not used and the query plan is the same as the original SQL without any new index built.

Let’s try the second composite index (EMP_GRADE, EMP_ADDRESS), the new query plan shows an Index Fast Full Scan of this index, it is a reasonable plan which no table’s data is needed to retrieve. So, the execution time is reduced from 16.83 seconds to 3.89 seconds.

Let’s test the last composite index (EMP_ADDRESS, EMP_GRADE) that EMP_ADDRESS is placed as the first column in the composite index, it creates a new query plan that shows an extra FIRST ROW operation for the INDEX FULL SCAN (MIN/MAX), it highly reduces the execution time from 16.83 seconds to 0.08 seconds.

So, indexing sometimes is an art that needs you to pay more attention to it, some potential solutions may perform excess your expectation.

The best index solution is now more than 200 times better than the original SQL without index, this kind of index recommendation can be achieved by Tosska SQL Tuning Expert for Oracle automatically.

https://tosska.com/tosska-sql-tuning-expert-pro-tse-pro-for-oracle/

Why You Must Speed Up Slow Queries in Oracle Database and SQL

oracle database performance tuning

The performance of an Oracle database and SQL query speed can directly affect the organisation it belongs to. If the queries running in the database are slow, they will surely have a negative impact. However, its severity may vary based on the database’s role, its architecture, and the industry your organisation operates in.

Regardless of the extent, it would be unwise to ignore them, which is why we are going to talk about all their effects in this blog.

How Slow Statements Affect Oracle Database and SQL Users

In this fast-paced world, everything needs to work fast and offer a quick response time to its end-users. The data that a web page displays generally comes directly from the database with very few interactions.

This implies the dependency of the application’s response time on the time it takes for the queries to run and the database to respond. Slow statements will take more time, resulting in loading screens before the desired information shows up. This is when Oracle database performance tuning becomes a requirement.

Such speeds don’t affect only the application, however; they leave an impact on the other parts of a system as well. The reason behind this is the location of the database in a majority of web architectures.

Take a look at the three-tier architecture, for example – the database lies at the bottom in most cases, forming the foundation. An increase in latency here is likely to cause the same in the higher levels along with other areas in the system.

Another way in which slow queries negatively impact the system is by making the database use more resources than is actually necessary. Some of these are available in limited quantities, such as I/O and CPU, since other applications share these resources.

On the other hand, not using existing resources sufficiently leads to their inefficient usage and slow queries as well. This may be the case with your database, so you may want to consider a few Oracle performance tuning tips that deal with this particular issue.

Top Reasons Behind Slow and Inefficient Queries

Given below are the three most significant causes of queries slowing down:

  1. Too many tasks: Executing a statement includes multiple tasks, such as retrieving data, making calculations, and arranging data in the order as the query specifies. All of these involve plenty of factors, any of which can increase the amount and complexity of work done, from joining and grouping to filtering and sorting.
  1. Too much waiting: Sometimes, statements don’t have too much to do, nor are they stuck waiting for resources. The reason why they are sluggish is that they are waiting on other statements that are locking resources or requiring higher levels of activity.
  1. Too few resources: Query execution works alongside other tasks taking place within a system. This means they share resources such as network throughput, disk I/O, and CPU. Statement execution is likely to take more time when these are completely occupied.

Locating and Working on Slow Queries in Oracle Database

Slow queries don’t get faster on their own – DBAs must take steps to speed them up. For starters, they can use the Database Performance Monitor (DPM) in the following ways:

  • Viewing all the queries that are taking up execution time using the query profiler. Such queries are often running in the absence of indexes, so it’s a good idea to add one to improve Oracle database performance and execution speed.
  • Automatically collecting explain plans to get a quick glance at the ones that contain information regarding slow queries and the changes related to them, if any. 
  • Assessing Oracle database and SQL to find out whether a statement can perform better with the help of some improvements.
  • Visiting the charts page to go through properly arranged metrics pertaining to system performance. This allows the DBA to set a threshold alert and note changes every time a system resource is reaching maximum use.

Conclusion

Based on your architecture and application, slow statements can affect more aspects of your business than just the database. Therefore, ignoring them is not recommended as it often results in a detrimental impact on both end-users and your organisation.

Consider enlisting the help of professional tuning tools to improve slow query performance in Oracle and SQL Server databases.

Transferring Data in SQL Server with an Eye on Performance

improve performance of sql query

A lot of database professionals often need to archive older data in SQL Server by transferring it from one table to another. There are multiple ways to achieve the transfer, the most useful of which we will discuss in this blog. We will also provide tips to ensure the performance of the database doesn’t get affected as these approaches are carried out.

Different Methods to Move Data from One Table to Another

Consider the following techniques that various DBAs take when they have to take data from a table to add to another table along with some ways to improve performance of SQL query while using them:

  1. Insert data with the INSERT INTO command – The INSERT INTO query is one of the basic methods of moving data from table 1 to table 2. You can help decrease the time it takes to enter information using this method. If the database is running under the full recovery model, just change it to the bulk-logged model. Doing this saves execution time as it skips over complete logging of bulk operations. The following query should help with this:

ALTER DATABASE <database name> SET RECOVERY <BULK_LOGGED>

Once you switch to the bulk-logged recovery model, you will have to use a truncate statement to flush table 2 (destination). You can carry out the same script you were using to transfer data after this.

  1. Use the SELECT INTO query – Using the SELECT INTO rather than the INSERT INTO command can prove useful in some cases. However, the benefits are significant when the recovery model is bulk-logged due to the reason mentioned above. Although users lack the ability to place the data in an existing table, SQL Server brought with it a feature to make things easier. It essentially enables them to pick the filegroup where they want to create a table.
  1. INSERT INTO query + Tab lock hint – Using both in combination has been known to provide better database performance. To achieve this, you will have to use TABLOCK for table 2. If the destination table is without a clustered index or other constraints, that data will remain as a heap. It helps to use the TABLOCK hint for the destination table during data insertion into a heap using the INSERT INTO statement. Doing this enhances query logging and locking since a shared lock is placed on the whole table rather than every row or page.
  2. Adding data using the SWITCH TO query – You can also try moving the data with the help of the SWITCH TO command. Although this query typically finds its use while transferring information between partitions among separate tables, it can help here as well. How? By moving data from one partition to the next using the ALTER TABLE command. If there are no allocated partitions, the data will transfer through tables instead. Before you begin data insertion, make sure you disable any constraints or indexes that exist on the table. It is better to enable constraints and rebuild indexes after insertion from a performance perspective.

Tips for Enhancing Performance During Data Transfer and Insertion

  • Reduce IO lag – Latency can negatively impact the process of writing database files on disk. You can decrease latency and bottlenecks using SSD drives that are comparatively better than SATA or SCSI drives.
  • Maintain Robust Server Infrastructure – The system needs to be properly built to ensure competent performance for various database operations. The greater the pressure on the resources, the greater the effect on performance.
  • Follow ACID Properties – ACID properties make sure each transaction contains certain properties when it gets processed. In the case of data insertion, the isolation factor is also important to consider because the values have another source. Here, the statements should contain the suitable isolation level to maintain integrity within the database.
  • Database Settings – One of the best ways to achieve improved outcomes is to maintain the right database configuration. This is because the settings can have a significant effect on performance. For instance, the location of the database files on the disk along with TempDB settings.

These are the various ways in which you can gain better performance at the query, trace, and constraint levels along with additions that can improve the execution of insert operations.

SQL Server: Knowing How Heaps and Clustered Indexes Work

MySQL database and sql

Heaps and clustered indexes are two different ways of storing data in SQL Server. Both have their advantages and disadvantages, and we will discuss them in this post.

A Bit about Heaps

Heaps are essentially piles of data that remain unsorted or unorganized, hence the name. Although you can find heaps on tables that don’t have clustered indexes, they may also be present with non-clustered indexes. Heaps provide the benefit of increased input speed which helps while adding data to a table. Data insertion is quicker because the process doesn’t require a logical order to do so.

A Bit about Clustered Indexes

A clustered index is a more organised way of data insertion. In fact, it is the go-to technique for logically sorting information in a table. A clustered index doesn’t need a primary key but you can create one on a predefined key-value. Most DBAs recommend creating them on the most-used columns that come under reference of highly frequent query executions. They also reduce the need for optimization since all the data gets sorted to fit them. The primary benefit of using a clustered index is that it speeds up data reads.

Knowing When to Use a Clustered Index

As noted above, using a clustered index leads to better read rates. Therefore, there are several instances where you may need to identify whether a clustered index will improve performance of SQL query rather than a heap.

To do this, you need to follow these steps:

  • First, it is important to understand where there is a requirement for greater read speed.
  • Check dynamic system views and look for large tables without a clustered index.
  • Once you locate a few such tables, you can analyse the plans and stats of queries in the MSSQL system dynamic management views. Searching through the table name in the variable will show you the usage frequency of the plan. It will also show the text fetched and other necessary validation details that show whether a heap or non-clustered index is in use instead.

You will be able to view object names in the second result set in case the table in question is under use in SQL object. Once you have reviewed the query plans relevant to the use cases, you will have sufficient information to help you decide whether the table requires a clustered index or if a heap is more suitable for it. You will also have to choose all the columns that will have to be in the index in case of the former. Tables with several use cases that mostly share the same columns can provide result sets faster with a clustered index.

When Not to Use Clustered Indexes

This is just as important to know because believe it or not, there are instances where a clustered index can do more harm than good to oracle database performance.

A logging table is one such instance as it normally has far more insert operations than reads or updates. This is because their purpose is to log each occurrence but users may not refer to it as frequently. If you place an index on this kind of table, it can result in hot latches due to lagging data insertions for the last available page. Meanwhile, information keeps getting added onto the same page from other means. The one case where this issue doesn’t occur is when the index’s main column is a GUID, therefore, it isn’t sequential.

Using a clustered index in a table with an excessive number of columns isn’t the best idea, either. The reason behind this is simple: the index is supposed to define the default sort order. Too many columns mean repeated resorting with each new use case, slowing down the database. It will also result in an increase in the size of the non-clustered indexes present in the table.

Another situation where a clustered index can’t help is a column that isn’t usually static as they undergo frequent changes. Changing key values on an index have far greater chances of creating performance-related problems. This is because updating key values typically leads to page splits – these need maintenance, which takes resources and affects performance.