Databases are designed to offer
an organized mechanism for storing, managing and retrieving information.
The definition of a database is a structured collection of records or data that is
stored in a computer system. In order for a database to be truly functional, it must not
only store large amounts of records well, but be accessed easily. [1]
In addition, new information and changes should also be fairly easy to input. In order to
have a highly efficient database system, you need to incorporate a program that manages the queries and information stored on the system. This
is usually referred to as DBMS or a Database Management System. Besides these features, all
databases that are created should be built with high data integrity and the ability to
recover data if hardware fails[2].
There are several common types of
databases; each type of database has its own data model (how the
data is structured).
A database model
or a database architecture is a
theory or specification describing how a database is structured and used. See also Database design.
- Big data
for
enterprises refers to the massive and complex datasets that organizations
collect and analyze to gain valuable insights and improve decision-making. These
datasets, often characterized by high volume, velocity, variety, and veracity (the 4
Vs), can include structured and unstructured data from various sources like customer
transactions, social media, and machine-generated data.
Here's a breakdown of key categories and a few examples,
and the specific tools for specific needs of the organization and
the type of data being processed:
Data Storage and Management -
Hadoop,
Cassandra,
MongoDB,
Snowflake,
Amazon Redshift,
Google BigQuery
Data Processing and Analytics -
Spark,
Kafka,
Pig,
Impala
Data Visualization and Business Intelligence -
Tableau,
Power BI,
Qlik,
Drill,
Domo
Other Important Tools
Talend,
RapidMiner,
KNIME,
Elasticsearch ,
Apache Storm,
ZooKeeper
See also:
What is BIG
DATA?,
Big data for
enterprise,
AWS Big Data
Data
Architecture
⇧
Data
Architecture basically deals with designing and constructing data resource. Data
Architecture provides methods to design, construct and implement a fully integrated,
business-driven data resource that include real world objects and events, onto appropriate
operating environments. Data Architecture also covers data resource components.
Followings are common database models:
- The Flat Model Database - In a flat model database, there is a two dimensional
(flat structure) array of data. For instance, there is one column of
information and within this column it is assumed that each data item will be
related to the other. For instance, a flat model database includes only zip
codes. Within the database, there will only be one column and each new row
within that one column will be a new zip code.
Flat File Tutorial,
FF Database Demo
- The Hierarchical Model Database - The hierarchical
model database resembles a tree like structure, such as how microsoft
windows organizes folders and files. In a hierarchical model database, each
upward link is nested in order to keep data organized in a particular order
on a same level list. For instance, a hierarchal database of sales, may list
each days sales as a separate file. Within this nested file are all of the
sales (same types of data) for the day.
See also:
Hierarchical Model,
Hierarchical Database Model,
for Web Developers,
Hierarchical Databases
- The Network Model - In a network model, the
defining feature is that a record is stored with a link to other records -
in effect networked. These networks (or sometimes referred to as pointers)
can be a variety of different types of information such as node numbers or
even a disk address.
See also:
The Network Database Model,
Network Database Model
|
- The
Relational Model - The relational model is the most popular type of
database and an extremely powerful tool, not only to store information, but
to access it as well. Relational databases are organized as tables.
Relational databases use a program interface
called SQL. IT/CS is
currently used on practically all relational databases. Relational databases
are extremely easy to customize to fit almost any kind of data storage. You
can easily create relations for items that you sell, employees that work for
your company, etc.
See also:
The Relational Model,
Relational Database tutorial,
Relational-Model
- Entity-relationship - The entity-relationship model
(or ER model) is a way of graphically representing the logical relationships
of entities (or objects) in order to create a database.
The ER model was first proposed by Peter Pin-Shan Chen of Massachusetts
Institute of Technology (MIT) in the 1970s.
See also:
ER Modeling,
Tutorial
- Object-relational model - The object-relational
model is designed to provide a relational database
management that allows developers to integrate databases with
their data types and methods. It is essentially a relational model that
allows users to integrate object-oriented features into it.
See also:
OR Model,
O/R Model,
Models,
ORDBMS
|
Comparision of models and more:
Comparing Database Types,
Optimized Architectures,
Compare
Models,
Data Model
Database management systems
⇧
One of the main functions of the database management system (DBMS) is doing the heavy lifting for
you. In other words, you don't necessarily have to know exactly where all that data is in
the system; as long as the database management system knows where it all is, it can deliver a
report for you to peruse. This might not seem to matter if you're thinking of just your
computer; but throw in a mainframe that contains reams and reams of data, and we're talking
about a huge amount of information that can be stored any number of places within the
mainframe system. The result is the same, though: a report that you can read, analyze, and
act on[2].
Database management systems frequently provide database server functionality. The back-end,
sometimes called a database server, performs tasks such as data analysis,
storage, data manipulation, archiving, and other non-user specific tasks. It helps to
specify the logical organization for a database and access and use the information within a
database. It provides facilities for controlling data access,
enforcing data
integrity, managing concurrency controlled, restoring database[3]. See also:
Types of DBMS,
List of DBMSs,
Comparison of DBMSs, Comparison of different SQL implementations,
some specific DBMS.
DBMS related topics:
Many of database concepts apply to all forms of
database management systems: relational (RDBMS), object (ODBMS), object-relational (ORDBMS), XML (XDBMS), and as well as object-relational
mapping (ORM) and XML-mapping (XM) products(tools).
|
|
- XML (XDBMS) - An XML-based DBMS is one that doesn't
worry about relations. It stores the data in the database in XML format.
See also:
XML and Data,
The Age of XML Database
- Object Relational Mapping (ORM) - allows one to
cleanly apply object-oriented design, analysis, and programming techniques
while hiding the specifics of dealing with the relational system.
See also:
Detailed,
ORM,
OR Oracle,
Mapping Tools,
OR Mapping Tool,
Frameworks
- XML Mapping - An object-relational mapping is
performed in two steps. First, an XML schema (e.g. DTD) is mapped to an
object schema, then the object schema is mapped to the database schema.
See also:
Mapping XML to Java,
XML Mapping,
Stylus Mapping Tools,
Altova Mapping Tools,
Castor
XML Mapping,
Microsoft,
Basic Mapping
|
More to read:
• Difference between a DBMS and RDBMS
• more RDBMS
• DBMS
• About SQL
• www.w3schools.com/sql
• Tutorialspoint
• SQL Tutorial
• SQL Advanced
Some examples of Database Management Systems (DBMS):
- MySQL -
Basically, it allows you to create a relational database structure on a
web-server somewhere in order to store data or automate procedures.
See more:
About,
with PHP,
Java JDBC,
MySQL Cluster
- FireBird, Interbase - is a relational database offering many ANSI
SQL-92 features.
See more:
About,
IBPhoenix,
Interbase,
Blogs,
Firebird Sql Server,
Interbase with JAVA
- SQL Server -
is designed for the enterprise environment. It runs on T-SQL.
See more:
Home 2008,
Overview,
with JAVA,
Tutorial
- Apache Derby (more) -
is a Java RDBMS that can be embedded in Java programs and used for online
transaction processing.
See:
About,
Java DB,
Using Java DB in Desktop Applications,
Java with Derby,
Connecting to a Derby
- Mongo DB
-
is an open-source document database, and the leading NoSQL
database, written in C++.
See more:
MongoDB,
Overview,
Java with MongoDB
|
- Oracle -
targets high-end workstations and minicomputers as the server platforms on
which to run its database systems. Along with Sun Microsystems, Oracle has
long been a champion of network computers.
See:
About,
Java,
Advanced Oracle and JAVA,
SQL Tutorial,
Pl/Sql
- XML
database - Xindice -
is a database designed from the ground up to store XML data or what is more
commonly referred to as a native XML database.
See:
Xindice
Home
- Sybase -
is a computer software company that develops and sells DBMS and middleware products
See more:
About
- DB2 -
which run on devices ranging from handhelds to mainframes, most often DB2
refers to the DB2 Enterprise Server Edition, which runs on Unix (AIX),
Windows, Linux and z/OS servers.
See more:
About,
DB Info,
Compare DB2 with SQL Server
- PostgreSQL - often simply Postgres, is an
open source object-relational database management system with an emphasis on
extensibility and standards compliance.
See more:
PostgreSQL,
Tutorials,
Java Practice,
psql -
PL/pgSQL
|
Some GUI Tools for DBMSs:
Other Tools:
• API's
• Programmer's Corner
• Software Development Tools
• Database Tools
• Modeling & Design
• Multimedia (AI Tools)
• Maths and Science
Database Types
⇧
- Relational Database - stores data in separate
tables instead of placing all data in one large table. When we talk about database, we mean a relational
database, in fact an RDBMS.
See also:
JDBC,
JPA
Frameworks,
Relational database,
??
- In-memory databases (IMDB) - is a memory-resident
relational database that eliminates disk access by storing and manipulating
data in main memory .
Hypersonic -
HSQLDB,
Derby,
Article
- No SQL Databases - database provides a mechanism for
storage and retrieval of data that is modeled in means other than the
tabular relations used in relational databases.
List Of NoSQL
Databases ,
Introduction,
Oracle NoSQL DB
MongoDB,
Monk,
mongodb
|
- Analytical database - provides access to all of the
data collected by an entity in interactive time. It transforms relational
database data. The relational database is denormalized (see. database normalization) and inverted such that the
data fields of tables in the relational database are stored in separate
files that contain a row number field and a single data field, see article.
Analytical database engine: OLAP, TM1,
OLAP
Cube
OLTP : OLTP,
OLTP
vs. OLAP,
Table design,
OLTP and DW
- Data warehouse - is a place where data is stored
for archival purposes, analysis purposes and security purposes.
See also:
DW,
Data Mining
API,
Star schema,
Normalization and Denormalization, Advanced Data Warehousing
- Types of NoSQL databases -
There are various approaches to classify NoSQL DBs, each with different
categories and subcategories, some of which overlap.
Elasticsearch,
Neo4j,
neo4j,
mongodb,
Spring Boot + Spring Data + Elasticsearch
example,
Java MongoDB Tutorial
|
Desktop Databases:
Desktop databases (e.g.
Microsoft
Access,
OpenOffice
Base,
Filemaker Pro,
Lotus Approach )
offer simple, flexible solutions for data storage and retrieval. They're often quite
sufficient to meet uncomplicated database requirements for both small and large
organizations.
Database Migration Tools
⇧
Flywaydb allows
version control for your database, robust schema evolution across all your environments and
ease, pleasure and plain SQL. Read more
Database
Migration With Flyway, How to use Flywaydb.
Liquibase (doc)
is an open source database-independent library for tracking, managing and applying database
schema changes. It was started in 2006 to allow easier tracking of database changes,
especially in an agile software development environment.
Read more about evolving your database using Spring-boot and
Liquibase,
and how to execute database migrations with
Liquibase (blog).
References
- Data Structures/Hash Tables
-
What is a Database?
- Database Server
- Data
Management
- Embedded
database
- eXtremeDB
in-memory database
|