Pages: [1]

Author Topic: Post Grabber  (Read 569 times)

0 Members and 1 Guest are viewing this topic.

Online Huntondoom

  • 0x13338
  • ******
  • Posts: 828
  • Karma: +18/-0
  • Gender: Male
  • Visual C# programmer
    • View Profile
Post Grabber
« on: November 01, 2011, 09:45:39 PM »
for a project Im working on right now I need to make a Post grabber, it needs to get the contents of a post (with username etc) in HTML

so far I have this:
Code: C
  1. string[] Posts = new string[0];
  2.             string Start = "<div class=\"windowbg\">".ToLower();
  3.                                                        
  4.             for (int S = 0; S < page.Length - Start.Length; ++S)
  5.             {
  6.                 string part = page.Substring(S, Start.Length).ToLower();
  7.                 if (part==Start)
  8.                 {
  9.                     int A = 0;
  10.                     for (int E = S; E < page.Length - 2; ++E)
  11.                     {
  12.                         part = page.Substring(E, 2);
  13.                         if (part.StartsWith("<") & !part.EndsWith("/")) {++A; }
  14.                         if (part == "</") { --A; }
  15.                         if (part == "</" & A == 0)
  16.                         {
  17.                             Posts = AddToArray(page.Substring(S, E - S), Posts);
  18.                             S = E;
  19.                             break;
  20.                         }
  21.                     }
  22.                 }
  23.             }
but is doesn't always give me the post or sometimes not enough
Logged
Aslong as you are connected to the internet, you'll have no privacy

Advanced Internet Search
Clean Up!

Online Kulverstukas

  • Administrator
  • 0x13338
  • *
  • Posts: 1928
  • Karma: +113/-10
  • Gender: Male
  • Delphi coder (and proud)
    • View Profile
    • My blog
Re: Post Grabber
« Reply #1 on: November 01, 2011, 10:25:52 PM »
Consider using Regex. Much more elegant and probably faster.
And is that Java?
Logged

Online Huntondoom

  • 0x13338
  • ******
  • Posts: 828
  • Karma: +18/-0
  • Gender: Male
  • Visual C# programmer
    • View Profile
Re: Post Grabber
« Reply #2 on: November 01, 2011, 11:25:39 PM »
Quote from: Kulverstukas on November 01, 2011, 10:25:52 PM
Consider using Regex. Much more elegant and probably faster.
And is that Java?
Visual C# which would be the mircosoft .NET version of java
Logged
Aslong as you are connected to the internet, you'll have no privacy

Advanced Internet Search
Clean Up!

Online Huntondoom

  • 0x13338
  • ******
  • Posts: 828
  • Karma: +18/-0
  • Gender: Male
  • Visual C# programmer
    • View Profile
Re: Post Grabber
« Reply #3 on: November 02, 2011, 03:01:33 PM »
but I dont want to be depend on Regex, cause its a suprise and I want it to work on everyones computer (everyone with Windows) without having them to upgrade their .net version
Logged
Aslong as you are connected to the internet, you'll have no privacy

Advanced Internet Search
Clean Up!

Online ande

  • Administrator
  • 0x13338
  • *
  • Posts: 1421
  • Karma: +81/-7
  • Gender: Male
    • View Profile
    • Evilzone
Re: Post Grabber
« Reply #4 on: November 02, 2011, 03:13:32 PM »
Quote from: Huntondoom on November 02, 2011, 03:01:33 PM
but I dont want to be depend on Regex, cause its a suprise and I want it to work on everyones computer (everyone with Windows) without having them to upgrade their .net version

Isn't regex a part of .net framework 2.0?
Logged

Online Huntondoom

  • 0x13338
  • ******
  • Posts: 828
  • Karma: +18/-0
  • Gender: Male
  • Visual C# programmer
    • View Profile
Re: Post Grabber
« Reply #5 on: November 02, 2011, 03:40:08 PM »
Quote from: ande on November 02, 2011, 03:13:32 PM
Isn't regex a part of .net framework 2.0?
dont know, I never use Regex
Logged
Aslong as you are connected to the internet, you'll have no privacy

Advanced Internet Search
Clean Up!

Online Kulverstukas

  • Administrator
  • 0x13338
  • *
  • Posts: 1928
  • Karma: +113/-10
  • Gender: Male
  • Delphi coder (and proud)
    • View Profile
    • My blog
Re: Post Grabber
« Reply #6 on: November 02, 2011, 03:45:01 PM »
Quote from: Huntondoom on November 02, 2011, 03:40:08 PM
dont know, I never use Regex
Well learn it. I did and now I couldn't live without it!
Logged

Online ande

  • Administrator
  • 0x13338
  • *
  • Posts: 1421
  • Karma: +81/-7
  • Gender: Male
    • View Profile
    • Evilzone
Re: Post Grabber
« Reply #7 on: November 02, 2011, 03:47:33 PM »
Quote from: Huntondoom on November 02, 2011, 03:40:08 PM
dont know, I never use Regex


Okay, found out. Infact, its a part of the 4.0 framework... Which sucks.
Logged

Online Huntondoom

  • 0x13338
  • ******
  • Posts: 828
  • Karma: +18/-0
  • Gender: Male
  • Visual C# programmer
    • View Profile
Re: Post Grabber
« Reply #8 on: November 02, 2011, 07:29:02 PM »
I made this so far, it now returns almost every post
Code: C
  1.  string[] Posts = new string[0];
  2.             string Start = "<div class=\"windowbg\">".ToLower();
  3.             string End = "<hr".ToLower();
  4.                                                        
  5.             for (int S = 0; S < page.Length - Start.Length; ++S)
  6.             {
  7.                 string part = page.Substring(S, Start.Length).ToLower();
  8.                 if (part==Start)
  9.                 {
  10.                     for (int E = S; E < page.Length - End.Length; ++E)
  11.                     {
  12.                         part = page.Substring(E, End.Length).ToLower();
  13.                         if (part==End)
  14.                         {
  15.                             Posts = AddToArray(page.Substring(S, E - S), Posts);
  16.                             S = E;
  17.                             break;
  18.                         }
  19.                     }
  20.                 }
  21.             }
  22.             return Posts;
Logged
Aslong as you are connected to the internet, you'll have no privacy

Advanced Internet Search
Clean Up!

Online Kulverstukas

  • Administrator
  • 0x13338
  • *
  • Posts: 1928
  • Karma: +113/-10
  • Gender: Male
  • Delphi coder (and proud)
    • View Profile
    • My blog
Re: Post Grabber
« Reply #9 on: November 02, 2011, 07:41:03 PM »
Kinda fucked up to read :D
When I was doing a client for an online dictionary, I didn't know regex at that time. If you want I can show you the extracting part. It's in Delphi though, but you may still get the algorithm idea.
Logged

Online Huntondoom

  • 0x13338
  • ******
  • Posts: 828
  • Karma: +18/-0
  • Gender: Male
  • Visual C# programmer
    • View Profile
Re: Post Grabber
« Reply #10 on: November 02, 2011, 08:34:46 PM »
Quote from: Kulverstukas on November 02, 2011, 07:41:03 PM
Kinda fucked up to read :D
When I was doing a client for an online dictionary, I didn't know regex at that time. If you want I can show you the extracting part. It's in Delphi though, but you may still get the algorithm idea.
sure but regex is .net 4.0 so im not going to use it :S
Logged
Aslong as you are connected to the internet, you'll have no privacy

Advanced Internet Search
Clean Up!

Online Kulverstukas

  • Administrator
  • 0x13338
  • *
  • Posts: 1928
  • Karma: +113/-10
  • Gender: Male
  • Delphi coder (and proud)
    • View Profile
    • My blog
Re: Post Grabber
« Reply #11 on: November 02, 2011, 09:16:10 PM »
Quote from: Huntondoom on November 02, 2011, 08:34:46 PM
sure but regex is .net 4.0 so im not going to use it :S
I said it doesn't use Regex.

Here is the code from few years ago :D

I used this to get an explanation for the word.
Code: [Select]
function pavyzdys(yKur : ansistring) : ansistring;
 var Pavyzdys : Record
                 Start,Endd : integer;
                end;
     i,m : integer;
begin
//====
 result := '';
//====
   if AnsiContainsText(yKur,'<p class=''pavyzdys''>') then
    begin
     Pavyzdys.Start   := AnsiPos('<p class=''pavyzdys''>',yKur);
     i := Pavyzdys.Start;
     m := 0;
      repeat
       Inc(i);
       m := m + 1
      until (AnsiPos(Copy(yKur,Pavyzdys.Start,m),'</p>') <> 0);
    end;
//====
  result := Copy(yKur,Pavyzdys.Start,m);
end;
Logged
Pages: [1]
 



Intern0t SoldierX py1337 SecurityOverride programisiai
Want to be here? Contact Ande or Satan911 on the forum or at IRC.