postgresql - Postgres COPY command - fields with commas, quoted with double quotes -


I searched and searched for some posts related to postgre CSV import, but nothing that resolves my current problem is.

I use the postgres copy command at all times to bring data from second source data sources to the current 100 million row Conflicts with CCV file, comma-delimited . The problem is with so many rows: 

  009098,0981098094, "something", "something else", "" this one, okay, there is a problem, "one more thing"   
 Fields enclosed in double quotes with embedded commas. Fields are not correctly parsed and I get an error:  
  "Error: additional data after the last expected column"   
 Generally when these I get ridiculous rows ad hoc, but this file is so big that I'm hoping to defend it from something else in a normal way. There is not a possibility to ask for a modified data format.  
  '/path/to/file.csv' csv header quote '' 'for example    
 
 
 
 This is distorted CSV.   
 has three values:  
     
  •     
  •   is "pancakes"  
  •   Home    
     The line you are having trouble with, repeats the double values:  
      009098,0981098094, "something", "something else", "" this one, okay, there is a problem "" another thing "^^ ^^   
     me It does not appear that anything in it can be as clear as the correct version is ambiguous: should it be  "it should be one, ok, a problem"  or is it  "" Is okay The problem is "" "?  
     
     I think you have to fix it by hand. A quick  sed  one-liner If you have a specific type of broken line , Then he should be able to work with it.  
     
     For reference purposes, I have seen the closest thing to the CSV standard and in section 2 it is to say:  
      5 Each field can be attached in double quotes (although some Ogram, like do not use Microsoft Excel, double quotes) if the area can not see inside are connected in double quotation marks, double quotation field. For example: "AAA", "BBB", "CCF", CRLF, Y, xxx [...] 7. If double-quotations are used to attach the field, then appear within a field A double quote giving will be saved before with another double quote. For example: "AAA", "B" "BB", "CCC"

Comments

Post a Comment

Popular posts from this blog

Python SQLAlchemy：AttributeError: Neither 'Column' object nor 'Comparator' object has an attribute 'schema' -

I tried to create a new database in my project, but when I run the script, I get this error, I have another project, this type of definition worked before, but now it gets a single error. I am using Python 2.7.8 and the version of SQLAlchemy module is 0.9.8. By the way, using a project flask- SQLAlchemy, it works well, I'm confused. Traceback information is as follows: Traceback (most recent call final): File "D: / Projects / IOM / db_create.py", line 4, & lt; Module & gt; From the models the base file "D: \ Projects \ OO-IM \ models.py", in line 15, in the & lt; Module & gt; Column ( 'followed_id', int (), ForeignKey ( 'user.id')) file "C: \ Python27 \ lib \ site-packages \ SQLAlchemy \ SQL \ schema.py", line 369, __new__ schema = metadata. Schema file "C: \ Python27 \ lib \ site-packages \ SQLAlchemy \ SQL \ elements.py", line 662, AttributeError in __getattr__ key): Neither 'column' object nor ...
Read more

java - How not to audit a join table and related entities using Hibernate Envers? -

I use Eners to be hibernate to audit my institutions. I have an audit entity, Foo , which includes a list & lt; Bars & gt; However, I do not want to audit the bar institutions, in the form of properties, I have written that: @inti @edicated public class Foo {@JoinTable (name = "T_FOO_BAR", joinColumns = @JoinColumn (name = "FOO_ID"), inverseJoinColumns = @JoinColumn (Name = "BAR_ID")) @ManyToMany (Cascade = Persist) @ Indited (targetAuditMode = RelationTargetAuditMode.NOT_AUDITED ) Public listing & lt; Bars & gt; GetBars () {Return bars; }} Now, I want to retrieve an amendment of foo : AuditReader reader = AuditReaderFactory.get (GetEntityManager ()); FU modification = (FU) reader.CreativeTeA () .entensEtrevision (Foo Class, 42) .getSingleResult (); Unfortunately, when I want to retrieve all the data (i.e. when it loads lazy times ), error me ORA-00942: table or view is not present , because Tried to query this: Sel...
Read more

mongodb - CakePHP paginator ignoring order, but only for certain values -

While pagging my user model in kppp, I can sort with some values, but not others. For example, I can order results from created or email , for example, but username or reputation seems to come back order by arbitrary command for example with a list of users: $ this-> paginate = array ('condition' => array ( 'User.is_active' = & gt; true), 'border' => 24, 'order' => array ('User.created' => DESC)); Works as expected, but $ this - & gt; Paged = array ('conditions' = & gt; array (' user.isIactive '= & gt; true),' border '= & gt; 24,' command '= & gt; array (' user repeats' = ' & Gt; DESC ')); No, no. I first thought that this could be a database issue, but when I search the database directly, sorted as expected. Note: I am using Mangodebi with the Mongo DB plugin of Ichikawa for the KPPHP; Pagnetizing users are used to work properl...
Read more