MySQL table design / architecture, table is too big -


I have a MySQL db that contains a lot of text, I'm in a data gathering from a website and a The table is inserted.

I am using SSD HD (100 GB) for DB and I could not find the place, I think that something in the structure of the table is too big, I can not guess the size For all the columns, I'm using the varchar \ text \ media text for most areas. When I insert all the data to DB, then I monitor the errors and when I see that I am trying to insert a particular field I am increasing the size of the area (like varchar (1000) to Varchar ( 2000)).

By now I have approximately 1.8 M lines, I think I am doing something wrong.

Here is the structure of my table << p> (1) default zero, `abstract` text,` foreign reference' value (15000) default zero, `currentUSClass` varchar (2200) default zero,` OtherReferences` mediumtext, `foreignPrio` varchar The default zero, `CurrentIntlClass` varchar (3000) default zero,` AppNum` varchar (45) default zero, `AppDate` date default zero,` Assignee` varchar (300) default zero, `inventors` varchar (1500) Default zero Invalid 'variant', 'ReferredBy` text, `AssigneeName` varchar (150),' Related User App ',' PrimaryExaminer ',' 100 'default', 'Assistant Examiner' varchar (100) default faucet,` AttorneyOrAgent` varchar (300) Default zero, `AssigneeState` varchar (80) Default zero;` AssigneeCity` varchar (150) Default zero; `InventorsName` varchar (800) Default zero;` InventorsState` varchar (300) Default zero; `InventorsCity` varchar (800) Default zero, `Claims` mediumtext,` Description` mediumtext, `InsertionTime` datetime not NULL, 'LastUpdatedOn` datetime not NULL, primary key (` pet id`), unique key' pat id_un IECE (`Pet Id`) Engine = MySam Default Charge = Latin 1;

What should I do? I have about 20% of the data (which means that I need 350GB) What is the display of space? Can I split the table into several tables on multiple tables? I am going to use the Sphinx in the index and in the end I am querying the data.

All non-item column values ​​stored in an 8KB record (undivided unit of space on your HDD) goes. Text column values ​​are stored in external blocks of data as pointers.

These types of structure (very text oriented) are better controlled by the NOSQL (SQL only) MongoDB database.

But I suspect there are many things that you can handle and amp;

The process of structuring a database (updating in one place - visible everywhere - is called generalization) is to be called normalization to avoid duplicate information and to allow easy updates.

If you have structured data such as those large VARCHARs (eg: the inventor's length 1500) as many elements of data (eg: names of different inventors by coma) then You can reorganize your DB, by making an inventory table and referencing the table.

Comments

Post a Comment

Popular posts from this blog

Python SQLAlchemy：AttributeError: Neither 'Column' object nor 'Comparator' object has an attribute 'schema' -

I tried to create a new database in my project, but when I run the script, I get this error, I have another project, this type of definition worked before, but now it gets a single error. I am using Python 2.7.8 and the version of SQLAlchemy module is 0.9.8. By the way, using a project flask- SQLAlchemy, it works well, I'm confused. Traceback information is as follows: Traceback (most recent call final): File "D: / Projects / IOM / db_create.py", line 4, & lt; Module & gt; From the models the base file "D: \ Projects \ OO-IM \ models.py", in line 15, in the & lt; Module & gt; Column ( 'followed_id', int (), ForeignKey ( 'user.id')) file "C: \ Python27 \ lib \ site-packages \ SQLAlchemy \ SQL \ schema.py", line 369, __new__ schema = metadata. Schema file "C: \ Python27 \ lib \ site-packages \ SQLAlchemy \ SQL \ elements.py", line 662, AttributeError in __getattr__ key): Neither 'column' object nor ...
Read more

java - How not to audit a join table and related entities using Hibernate Envers? -

I use Eners to be hibernate to audit my institutions. I have an audit entity, Foo , which includes a list & lt; Bars & gt; However, I do not want to audit the bar institutions, in the form of properties, I have written that: @inti @edicated public class Foo {@JoinTable (name = "T_FOO_BAR", joinColumns = @JoinColumn (name = "FOO_ID"), inverseJoinColumns = @JoinColumn (Name = "BAR_ID")) @ManyToMany (Cascade = Persist) @ Indited (targetAuditMode = RelationTargetAuditMode.NOT_AUDITED ) Public listing & lt; Bars & gt; GetBars () {Return bars; }} Now, I want to retrieve an amendment of foo : AuditReader reader = AuditReaderFactory.get (GetEntityManager ()); FU modification = (FU) reader.CreativeTeA () .entensEtrevision (Foo Class, 42) .getSingleResult (); Unfortunately, when I want to retrieve all the data (i.e. when it loads lazy times ), error me ORA-00942: table or view is not present , because Tried to query this: Sel...
Read more

mongodb - CakePHP paginator ignoring order, but only for certain values -

While pagging my user model in kppp, I can sort with some values, but not others. For example, I can order results from created or email , for example, but username or reputation seems to come back order by arbitrary command for example with a list of users: $ this-> paginate = array ('condition' => array ( 'User.is_active' = & gt; true), 'border' => 24, 'order' => array ('User.created' => DESC)); Works as expected, but $ this - & gt; Paged = array ('conditions' = & gt; array (' user.isIactive '= & gt; true),' border '= & gt; 24,' command '= & gt; array (' user repeats' = ' & Gt; DESC ')); No, no. I first thought that this could be a database issue, but when I search the database directly, sorted as expected. Note: I am using Mangodebi with the Mongo DB plugin of Ichikawa for the KPPHP; Pagnetizing users are used to work properl...
Read more