What is Sysbench?
Sysbench is designed for evaluating the performance of systems by executing CPU, memory, file I/O, and database benchmarks. It is highly customizable, and the results can be used for performance tuning, comparing different configurations, or validating hardware upgrades.
Installing Sysbench on CentOS and Ubuntu.
Step 1: Install Dependencies
Run the following command to install the required dependencies:
sudo yum install mysql-devel libtool automake autoconf
Step 2: Download Sysbench
For example, to download Sysbench from GitHub, you can use curl:
curl -LO https://github.com/akopytov/sysbench/releases/download/1.0.20/sysbench-1.0.20.tar.gz
If the .tar.gz file is downloaded, extract it:
tar -xzvf sysbench-1.0.20.tar.gz
cd sysbench-1.0.20/
Step 3: Build Sysbench from Source
./autogen.sh ; ./configure ; make -j ; make install
Step 4: Verify the Installation
sysbench --version
sysbench 1.0.20
Running the CPU Benchmark Test:
Sysbench provides a number of tests, but here, we will focus on testing the CPU performance. Sysbench’s CPU test calculates prime numbers up to a specified limit and reports the number of events processed during the benchmark run.
Here’s how you can run the benchmark on the CPU:
Running Sysbench CPU Test with 1 Thread:
sysbench --test=cpu --threads=1 --time=60 run
Sample Output:
WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
sysbench 1.0.20 (using bundled LuaJIT 2.1.0-beta2)
Running the test with following options:
Number of threads: 1
Initializing random number generator from current time
Prime numbers limit: 10000
Initializing worker threads...
Threads started!
CPU speed:
events per second: 2955.46
General statistics:
total time: 60.0002s
total number of events: 177333
Latency (ms):
min: 0.33
avg: 0.34
max: 0.39
95th percentile: 0.35
sum: 59978.41
Threads fairness:
events (avg/stddev): 177333.0000/0.00
execution time (avg/stddev): 59.9784/0.00
Running Sysbench CPU Test with 2 Threads:
sysbench --test=cpu --threads=2 --time=60 run
Sample Output:
CPU speed:
events per second: 5903.08
General statistics:
total time: 60.0002s
total number of events: 354195
Latency (ms):
min: 0.33
avg: 0.34
max: 1.29
95th percentile: 0.35
sum: 119915.68
Threads fairness:
events (avg/stddev): 177097.5000/89.50
execution time (avg/stddev): 59.9578/0.00
As you can see, with more threads, the events per second increase as well, which indicates better CPU utilization.
Running Sysbench CPU Test with 4 Threads:
Increasing the number of threads even more tests the system's scalability:
sysbench --test=cpu --threads=4 --time=60 run
Sample Output:
CPU speed:
events per second: 11819.27
General statistics:
total time: 60.0003s
total number of events: 709178
Latency (ms):
min: 0.33
avg: 0.34
max: 1.28
95th percentile: 0.35
sum: 239860.45
Threads fairness:
events (avg/stddev): 177294.5000/122.97
execution time (avg/stddev): 59.9651/0.00
Running Sysbench CPU Test with 8 Threads:
sysbench --test=cpu --threads=8 --time=60 run
Sample Output:
CPU speed:
events per second: 23637.61
General statistics:
total time: 60.0004s
total number of events: 1418301
Latency (ms):
min: 0.33
avg: 0.34
max: 1.28
95th percentile: 0.35
sum: 479730.48
Threads fairness:
events (avg/stddev): 177287.6250/42.17
execution time (avg/stddev): 59.9663/0.00
Running Sysbench CPU Test with 16 Threads:
sysbench --test=cpu --threads=16 --time=60 run
Sample Output:
CPU speed:
events per second: 47267.52
General statistics:
total time: 60.0004s
total number of events: 2836140
Latency (ms):
min: 0.33
avg: 0.34
max: 1.42
95th percentile: 0.35
sum: 959459.02
Threads fairness:
events (avg/stddev): 177287.6250/42.17
execution time (avg/stddev): 59.9662/0.00
Observations:
Threads and Performance: As the number of threads increases, the events per second increase proportionally. This shows how well your CPU can handle multi-threaded workloads.
Latency: The latency remains fairly constant across different thread counts, which suggests that the CPU's ability to process each event is relatively consistent.
Thread Fairness: The fairness remains stable, indicating that Sysbench is distributing tasks evenly across the threads.
Friday, 27 December 2024
Friday, 22 November 2024
Prometheus vs InfluxDB: Choosing the Best Time-Series Database for Monitoring
When it comes to monitoring the performance and health of your applications, systems, and infrastructure, time-series data plays a key role. A time-series database is essential for managing and analyzing this data effectively.
InfluxDB and Prometheus are two of the most popular open-source tools for handling time-series data. They are both widely used, but each serves a different purpose and has its own advantages. InfluxDB has a broad range of time-series data storage capabilities, including system metrics and IoT data, while Prometheus is popular for monitoring real-time metrics and cloud-native environments.
What Is Prometheus?
Prometheus is an open-source monitoring and alerting toolkit developed by SoundCloud and later contributed to the Cloud Native Computing Foundation (CNCF). It is widely adopted for monitoring the health of applications, microservices, containers, and infrastructure, particularly in Kubernetes-based environments.
Prometheus collects and stores metrics in a time-series format, where each time-series is identified by a metric name and associated labels (key-value pairs). Prometheus uses a pull-based model to scrape data from various sources like application endpoints, servers, or exporters.
Key Features of Prometheus:
Pull-based model: Prometheus scrapes metrics from configured endpoints, which allows for a decentralized and flexible architecture.
PromQL: A powerful query language designed specifically for time-series data. PromQL allows for aggregating, filtering, and visualizing metrics.
Alerting: Built-in alerting capabilities through Alertmanager, enabling users to define alert rules based on metric values.
Data retention: Prometheus stores data on disk using a custom, time-series optimized format and allows you to configure retention periods manually.
Integration with Grafana: Prometheus integrates seamlessly with Grafana to visualize metrics on customizable dashboards.
What Is InfluxDB?
InfluxDB is another popular open-source time-series database developed by InfluxData. Unlike Prometheus, which is primarily focused on monitoring and alerting, InfluxDB is a more general-purpose time-series database that can handle various types of time-series data, including metrics, events, logs, and IoT data.
InfluxDB follows a push-based model, where data is written to the database using an HTTP API or other ingestion methods like Telegraf (an open-source agent for collecting, processing, and sending metrics).
Key Features of InfluxDB:
Push-based model: Data is pushed to InfluxDB either via its API or through Telegraf agents, making it suitable for scenarios where the data is generated by external systems or devices.
InfluxQL and Flux: InfluxDB uses InfluxQL, a SQL-like query language, for querying time-series data. Flux is a more powerful, functional query language that enables complex transformations, aggregations, and analytics.
Continuous queries: InfluxDB supports continuous queries to automatically downsample and aggregate data over time, making it ideal for long-term data retention and historical analysis.
Retention policies: InfluxDB allows users to define automatic retention policies, meaning older data can be automatically dropped or downsampled as needed.
Clustering and High Availability: InfluxDB Enterprise provides support for clustering, data replication, and high availability (HA), enabling horizontal scaling for large-scale environments.
Integration with Grafana: Like Prometheus, InfluxDB integrates with Grafana for visualizing time-series data on interactive dashboards.
Prometheus vs InfluxDB: A Detailed Comparison
DBMS_LOB Functions in Oracle
Managing Large Objects (LOBs) like text, images, audio, or video requires special tools. Oracle provides the DBMS_LOB package to work with these large objects efficiently. This package includes functions that allow developers to read, write, search, copy, and modify LOBs (CLOB, BLOB, and NCLOB).
Key DBMS_LOB Functions:
1. DBMS_LOB.GETLENGTH
DBMS_LOB.GETLENGTH function returns the length of a LOB in bytes (for BLOB) or characters (for CLOB/NCLOB). It’s useful when you want to find the size of the data in the LOB.
Example: Let’s assume you have a CLOB column document_content in the documents table. You want to find out the length of the content in a specific document (with document_id = 101).
SELECT DBMS_LOB.GETLENGTH(document_content) AS doc_length
FROM documents
WHERE document_id = 101;
This will return the length of the document_content LOB.
2. DBMS_LOB.INSTR
DBMS_LOB.INSTR function searches for a substring within a LOB. It returns the position of the substring or 0 if not found.
Example: Suppose you want to find the position of the word "Oracle" in the document_content LOB of the documents table for document_id = 101.
SELECT DBMS_LOB.INSTR(document_content, 'Oracle') AS position
FROM documents
WHERE document_id = 101;
This will return the position of the first occurrence of the word "Oracle" within the document_content LOB.
3. DBMS_LOB.COPY
DBMS_LOB.COPY function copies data from one LOB to another. This is useful for duplicating or transferring LOB data.
Example: You want to copy the document_content from one document to another in the same documents table.
DECLARE
v_source_lob CLOB;
v_dest_lob CLOB;
BEGIN
-- Retrieve source LOB
SELECT document_content
INTO v_source_lob
FROM documents
WHERE document_id = 101;
-- Retrieve destination LOB
SELECT document_content
INTO v_dest_lob
FROM documents
WHERE document_id = 102;
-- Copy data from source to destination
DBMS_LOB.COPY(v_dest_lob, v_source_lob, DBMS_LOB.GETLENGTH(v_source_lob));
COMMIT;
END;
/
This copies the content from one LOB to another, ensuring the entire content is transferred.
4. DBMS_LOB.ERASE
DBMS_LOB.ERASE function erases a portion of a LOB. It’s useful when you need to delete part of the data from a LOB.
Example: Let’s say you want to erase the first 100 bytes of a BLOB stored in the image_data column of the images table.
DECLARE
v_blob BLOB;
BEGIN
-- Retrieve the BLOB
SELECT image_data
INTO v_blob
FROM images
WHERE image_id = 1001;
-- Erase the first 100 bytes of the BLOB
DBMS_LOB.ERASE(v_blob, 100);
COMMIT;
END;
/
This will erase the first 100 bytes of the image_data BLOB.
5. DBMS_LOB.SUBSTR
DBMS_LOB.SUBSTR function allows you to extract a substring from a LOB. You can define the starting position and the length of the substring.
Example: If you want to retrieve the first 100 characters of a CLOB column document_content for document_id = 101:
SELECT DBMS_LOB.SUBSTR(document_content, 100, 1) AS first_100_chars
FROM documents
WHERE document_id = 101;
This extracts the first 100 characters from the document_content LOB starting from position 1.
6. DBMS_LOB.COMPARE
DBMS_LOB.COMPARE function compares two LOBs. It returns:
A value less than 0 if the first LOB is smaller.
0 if the LOBs are equal.
A value greater than 0 if the first LOB is larger.
Example: You want to compare the document_content of two documents with document_id = 101 and document_id = 102.
SELECT DBMS_LOB.COMPARE(doc1.document_content, doc2.document_content) AS comparison_result
FROM documents doc1, documents doc2
WHERE doc1.document_id = 101 AND doc2.document_id = 102;
This compares the two document_content LOBs and returns the result.
7. DBMS_LOB.APPEND
DBMS_LOB.APPEND function appends one LOB to the end of another. It’s helpful when you want to add content to an existing LOB.
Example: Suppose you want to append content from one document (document_id = 103) to another (document_id = 101).
DECLARE
v_additional_content CLOB;
v_existing_content CLOB;
BEGIN
-- Retrieve the additional content to append
SELECT additional_content
INTO v_additional_content
FROM documents
WHERE document_id = 103;
-- Retrieve the existing content of document 101
SELECT document_content
INTO v_existing_content
FROM documents
WHERE document_id = 101;
-- Append the additional content
DBMS_LOB.APPEND(v_existing_content, v_additional_content);
COMMIT;
END;
/
This appends the additional_content to the document_content of document 101.
8. DBMS_LOB.READ
DBMS_LOB.READ function allows you to read a specified portion of a LOB. You can define the starting position and the number of bytes (for BLOBs) or characters (for CLOBs/NCLOBs) to read.
Example: Let’s read the first 50 bytes of a BLOB from the image_data column in the images table.
DECLARE
v_image_data BLOB;
v_read_data VARCHAR2(50);
BEGIN
-- Retrieve the BLOB data
SELECT image_data
INTO v_image_data
FROM images
WHERE image_id = 1001;
-- Read the first 50 bytes
v_read_data := DBMS_LOB.READ(v_image_data, 50, 1);
-- Output the read data (in hexadecimal format)
DBMS_OUTPUT.PUT_LINE(v_read_data);
END;
/
This reads the first 50 bytes of the image_data BLOB starting from position 1.
9. DBMS_LOB.WRITEAPPEND
DBMS_LOB.WRITEAPPEND function allows you to append data to the end of a LOB, such as adding more text to an existing CLOB.
Example: Suppose you want to append the string "Additional data" to the document_content LOB for document_id = 101:
DECLARE
v_append_data VARCHAR2(100) := 'Additional data';
v_document_content CLOB;
BEGIN
-- Retrieve the document content
SELECT document_content
INTO v_document_content
FROM documents
WHERE document_id = 101;
-- Append the data
DBMS_LOB.WRITEAPPEND(v_document_content, LENGTH(v_append_data), v_append_data);
COMMIT;
END;
/
This appends the string "Additional data" to the document_content LOB.
Saturday, 16 November 2024
HAProxy Log Rotation Not Working? Here’s How to Fix It
When running HAProxy in production, it's crucial that log files are rotated properly to prevent excessive disk usage and system slowdowns. If HAProxy logs are not rotating as expected, it could lead to your disk filling up, affecting the performance and reliability of your system.
If your HAProxy logs are not rotating, it could be due to several possible reasons.
In this post, we'll walk through the most common causes of log rotation issues, how to troubleshoot them, and provide a real-world use case with a solution.
HAProxy typically uses logrotate to handle log file rotation. If your log files are not rotating, it could be due to a missing or misconfigured logrotate configuration.
How to Check Logrotate Configuration:
Ensure there is a logrotate configuration file for HAProxy in /etc/logrotate.d/
It should look similar to the following:
/var/log/haproxy.log {
daily
missingok
rotate 7
compress
notifempty
create 0640 haproxy adm
sharedscripts
postrotate
/etc/init.d/haproxy reload > /dev/null 2>/dev/null || true
endscript
}
Explanation of Directives:
daily: Rotate the log files daily. You can also use weekly, monthly, etc., depending on your requirements.
rotate 7: Keep 7 backup log files before deleting the oldest.
compress: Compress old log files to save disk space.
create 0640 haproxy adm: This ensures that new log files are created with proper permissions (0640), and the owner is set to haproxy, with the group as adm.
postrotate: This ensures that HAProxy is reloaded after log rotation to begin writing to the new log file. If HAProxy is still writing to the old log file, logrotate will not be able to rename the rotated file.
Troubleshooting:
If the logrotate configuration is missing or incorrectly configured, you can either create or update the configuration file as shown above.
To check if logrotate is working correctly, run the following command to simulate the log rotation process:
sudo logrotate -d /etc/logrotate.conf
This command will display what logrotate would do, but will not actually rotate any logs. This is useful for troubleshooting.
2. Permissions Issues
If the HAProxy log files are not being written to or rotated due to permission issues, you need to verify that HAProxy has write access to its log file and the directory.
Check the permissions of /var/log/haproxy.log and ensure the user HAProxy runs as (usually haproxy) has the correct permissions:
ls -l /var/log/haproxy.log
Check that the logrotate user (usually root) has the necessary permissions to rotate the file.
If permissions are incorrect, adjust them with chown and chmod:
sudo chown haproxy:adm /var/log/haproxy.log
sudo chmod 0640 /var/log/haproxy.log
3. Log Output Configuration in HAProxy
HAProxy must be configured to log to a file (e.g., /var/log/haproxy.log). Ensure your HAProxy configuration includes proper logging directives:
In /etc/haproxy/haproxy.cfg, make sure you have something like the following:
global
log /dev/log local0
defaults
log global
option httplog
This tells HAProxy to log to the syslog facility local0, which is often associated with the HAProxy logs. If this is not set correctly, HAProxy may not be logging to the expected location.
4. Logfile Being Open by HAProxy Process
If the HAProxy process is holding the log file open (e.g., if HAProxy is still running with the old log file after rotation), logrotate might fail to rename the file. You can ensure that HAProxy is properly reloading by sending a SIGHUP signal to HAProxy, or by using the postrotate script in the logrotate config (mentioned above).
To manually reload HAProxy, you can:
sudo systemctl reload haproxy
or
sudo service haproxy reload
5. Logrotate Not Running
If logrotate is not running automatically (e.g., if the cron job for logrotate is not configured or working), the logs will not rotate.
Check cron jobs: Ensure that the logrotate cron job is enabled. You can check cron jobs by listing them with:
crontab -l
Alternatively, check if the logrotate service is running (on systems that use systemd):
systemctl status logrotate
To test logrotate manually, run:
sudo logrotate /etc/logrotate.conf
6. Disk Space Issues
If your disk is full, logrotate may not be able to create new log files or rotate old ones. You can check disk usage with:
df -h
If the disk is full, free up some space or increase the disk size.
Monday, 28 October 2024
Kubernetes Engine (OKE) supports the following versions of Kubernetes for new clusters
Container Engine for Kubernetes now supports Kubernetes version 1.28.10, in addition to versions 1.30.1 and 1.29.1. Please be aware that with the addition of support for version 1.28.10, Container Engine for Kubernetes will officially discontinue support for version 1.28.2 starting October 8, 2024.
Kubernetes Engine (OKE) supports the following versions of Kubernetes for new clusters:
Ref: https://docs.oracle.com/en-us/iaas/releasenotes/conteng/conteng-K8s-1-28-10-support.htm
Tuesday, 27 August 2024
Understanding and Fixing the IndentationError: expected an indented block after 'if' statement in Python
Python is renowned for its clean and readable syntax, which includes its use of indentation to define blocks of code. One common issue beginners (and sometimes even experienced developers) encounter is the IndentationError: expected an indented block after 'if' statement. This error can be frustrating, but understanding its cause and how to fix it can save a lot of time and hassle.
In this blog, we’ll explore what triggers this error and how to resolve it effectively.
What is the IndentationError: expected an indented block after 'if' statement?
The IndentationError in Python occurs when the interpreter expects an indented block of code but doesn’t find one. Specifically, when you use control structures like if, for, while, or try, Python expects an indented block following these statements to define the scope of the code that should be executed. If this indentation is missing, Python raises an IndentationError.
Example of the Error
x = 10
if x > 5:
print("x is greater than 5")
When you run this code, Python will throw an IndentationError like this:
ERROR!
Traceback (most recent call last):
File "<main.py>", line 3
print("x is greater than 5")
^^^^^
IndentationError: expected an indented block after 'if' statement on line 2
=== Code Exited With Errors ===
Why Does This Error Occur?
In Python, indentation is used to define blocks of code. For example, after an if statement, you need to indent the code that should execute if the condition is true. This indentation can be spaces or tabs, but it must be consistent. The error occurs because Python expects an indented block to follow the if statement, and if it doesn’t find it, it raises an error.
How to Fix the Error
To resolve this error, you need to provide an indented block of code following the if statement. Here’s how you can correct the example:
x = 10
if x > 5:
print("x is greater than 5")
In this corrected version, the print statement is indented with four spaces (or a tab, depending on your editor's configuration). This indentation indicates that print("x is greater than 5") is part of the if block and should be executed when x > 5 evaluates to True.
Best Practices for Indentation in Python
Consistent Indentation: Use either spaces or tabs for indentation, but do not mix them. The Python community typically prefers 4 spaces per indentation level.
Editor Configuration: Most modern code editors and IDEs can be configured to automatically insert spaces when you press the tab key, which helps avoid mixing tabs and spaces.
Check Indentation: If you encounter indentation errors, double-check that all blocks of code are properly indented and that your editor is set up correctly.
Use Linters: Tools like flake8 or pylint can help catch indentation issues and other coding standards violations before you run your code.
Subscribe to:
Posts (Atom)