In [1]:
import re
import requests
from bs4 import BeautifulSoup

Top 100 크롤링

In [2]:
top100_url="http://www.melon.com/chart/index.htm"
In [3]:
html=requests.get(top100_url).text
soup=BeautifulSoup(html,'html.parser')
In [7]:
for tag in soup.select('#tb_list a[href*=playSong]'):
    title=tag.text
    js=tag['href']
    matched=re.search(r"'(\d+)'\);",js)
    if matched:
        #print(matched)
        song_id=matched.group(1)
        song_url="http://www.melon.com/song/detail.htm?songId="+song_id
        print(title,song_url)
SIGNAL http://www.melon.com/song/detail.htm?songId=30413326
I LUV IT http://www.melon.com/song/detail.htm?songId=30403589
New Face http://www.melon.com/song/detail.htm?songId=30403590
맞지? http://www.melon.com/song/detail.htm?songId=30408193
오늘 취하면 (Feat.창모) (Prod. SUGA) http://www.melon.com/song/detail.htm?songId=30390068
팔레트 (Feat. G-DRAGON) http://www.melon.com/song/detail.htm?songId=30378156
REALLY REALLY http://www.melon.com/song/detail.htm?songId=30341745
Shape of You http://www.melon.com/song/detail.htm?songId=30188113
BLUE MOON (Prod. GroovyRoom) http://www.melon.com/song/detail.htm?songId=30367365
사랑이 잘 (With 오혁) http://www.melon.com/song/detail.htm?songId=30349593
TOMBOY http://www.melon.com/song/detail.htm?songId=30383949
밤편지 http://www.melon.com/song/detail.htm?songId=30314784
Marry Me http://www.melon.com/song/detail.htm?songId=4786735
첫눈처럼 너에게 가겠다 http://www.melon.com/song/detail.htm?songId=30190630
KNOCK KNOCK http://www.melon.com/song/detail.htm?songId=30261007
미치고 싶다 http://www.melon.com/song/detail.htm?songId=30399360
아름다워 http://www.melon.com/song/detail.htm?songId=8276709
혼자 http://www.melon.com/song/detail.htm?songId=30419668
좋다고 말해 http://www.melon.com/song/detail.htm?songId=30163110
얼굴 찌푸리지 말아요 http://www.melon.com/song/detail.htm?songId=30310862
She`s a Baby http://www.melon.com/song/detail.htm?songId=30355842
이런 엔딩 http://www.melon.com/song/detail.htm?songId=30378157
오빠야 http://www.melon.com/song/detail.htm?songId=5605860
이 지금 http://www.melon.com/song/detail.htm?songId=30378155
봄날 http://www.melon.com/song/detail.htm?songId=30244931
마지막 장면 (Feat. 이성경) http://www.melon.com/song/detail.htm?songId=30403591
All I Wanna Do (K) (Feat. Hoody, Loco) http://www.melon.com/song/detail.htm?songId=30049271
마에스트로 (Maestro) http://www.melon.com/song/detail.htm?songId=8276708
부담이 돼 (Feat. 휘인 Of 마마무) http://www.melon.com/song/detail.htm?songId=30290011
팩트폭행 (Feat. G-DRAGON) http://www.melon.com/song/detail.htm?songId=30403595
오랜 날 오랜 밤 http://www.melon.com/song/detail.htm?songId=30181278
우주를 줄게 http://www.melon.com/song/detail.htm?songId=9620469
TT http://www.melon.com/song/detail.htm?songId=30054344
불장난 http://www.melon.com/song/detail.htm?songId=30069773
나만 안되는 연애 http://www.melon.com/song/detail.htm?songId=9620473
잼잼 http://www.melon.com/song/detail.htm?songId=30378159
처음부터 너와 나 http://www.melon.com/song/detail.htm?songId=30414090
CHEER UP http://www.melon.com/song/detail.htm?songId=8158031
Rookie http://www.melon.com/song/detail.htm?songId=30227054
Closer (Feat. Halsey) http://www.melon.com/song/detail.htm?songId=8287788
그대라는 사치 http://www.melon.com/song/detail.htm?songId=9614761
에라 모르겠다 http://www.melon.com/song/detail.htm?songId=30147445
BOMB (Feat. B.I, BOBBY) http://www.melon.com/song/detail.htm?songId=30403593
D (half moon) (Feat. 개코) http://www.melon.com/song/detail.htm?songId=8109412
Something Just Like This http://www.melon.com/song/detail.htm?songId=30271730
Fine http://www.melon.com/song/detail.htm?songId=30278209
심술 http://www.melon.com/song/detail.htm?songId=8155478
Beautiful http://www.melon.com/song/detail.htm?songId=30157753
돌아오지마 (Feat. 용준형 Of 비스트) http://www.melon.com/song/detail.htm?songId=8143111
YESTERDAY http://www.melon.com/song/detail.htm?songId=30232778
Love Yourself http://www.melon.com/song/detail.htm?songId=7931286
이 소설의 끝을 다시 써보려 해 http://www.melon.com/song/detail.htm?songId=5398990
어디에도 http://www.melon.com/song/detail.htm?songId=8033528
Decalcomanie (데칼코마니) http://www.melon.com/song/detail.htm?songId=30079086
너였다면 http://www.melon.com/song/detail.htm?songId=8203900
우리집을 못 찾겠군요 (Feat. 볼빨간사춘기) http://www.melon.com/song/detail.htm?songId=30230986
나야 나 (PICK ME) http://www.melon.com/song/detail.htm?songId=30294022
LOVE (Feat. 태양) http://www.melon.com/song/detail.htm?songId=30403592
You(=I) http://www.melon.com/song/detail.htm?songId=9620471
Stay With Me http://www.melon.com/song/detail.htm?songId=30132687
MOVIE http://www.melon.com/song/detail.htm?songId=30286429
바람이 불었으면 좋겠어 http://www.melon.com/song/detail.htm?songId=4383833
비가와 (Rain) http://www.melon.com/song/detail.htm?songId=30247520
숨 http://www.melon.com/song/detail.htm?songId=30011624
너란 봄 (Feat. 하림) http://www.melon.com/song/detail.htm?songId=30351579
이름에게 http://www.melon.com/song/detail.htm?songId=30378164
내가 저지른 사랑 http://www.melon.com/song/detail.htm?songId=9631530
낮보다는 밤 http://www.melon.com/song/detail.htm?songId=30349960
See You Again (폴 워커 추모 엔딩곡) http://www.melon.com/song/detail.htm?songId=5629279
아프지 마요 http://www.melon.com/song/detail.htm?songId=30394259
바래다줄게 (Take You Home) http://www.melon.com/song/detail.htm?songId=30367190
FINGERTIP http://www.melon.com/song/detail.htm?songId=30285450
널 사랑하지 않아 http://www.melon.com/song/detail.htm?songId=8199190
피 땀 눈물 http://www.melon.com/song/detail.htm?songId=30029173
도원경 (桃源境) http://www.melon.com/song/detail.htm?songId=30413431
문득 http://www.melon.com/song/detail.htm?songId=30414362
남자의 정석 (Feat. 보이비) (Prod. 길 Of 매직맨션) http://www.melon.com/song/detail.htm?songId=30425380
Make Me Love You http://www.melon.com/song/detail.htm?songId=30343207
그러나, 밤 http://www.melon.com/song/detail.htm?songId=30416232
Black Out http://www.melon.com/song/detail.htm?songId=30378160
러시안 룰렛 (Russian Roulette) http://www.melon.com/song/detail.htm?songId=9634954
노래 http://www.melon.com/song/detail.htm?songId=30227061
이쁘다니까 http://www.melon.com/song/detail.htm?songId=30169608
Dejavu http://www.melon.com/song/detail.htm?songId=30403070
넘어와 (Feat. 백예린) http://www.melon.com/song/detail.htm?songId=30257703
지금, 우리 http://www.melon.com/song/detail.htm?songId=30395923
저 별 http://www.melon.com/song/detail.htm?songId=30132701
I`ll Be Yours http://www.melon.com/song/detail.htm?songId=30324427
그대로일까 (Feat. 헤이즈) http://www.melon.com/song/detail.htm?songId=30394657
소나기 http://www.melon.com/song/detail.htm?songId=30208070
Be There http://www.melon.com/song/detail.htm?songId=30421827
너무너무너무 http://www.melon.com/song/detail.htm?songId=30040810
I Miss You http://www.melon.com/song/detail.htm?songId=30177818
이 바보야 http://www.melon.com/song/detail.htm?songId=30120986
I`m Not The Only One http://www.melon.com/song/detail.htm?songId=4660794
슬픈 노래 http://www.melon.com/song/detail.htm?songId=30394260
기다렸다 가 http://www.melon.com/song/detail.htm?songId=30215199
봄 사랑 벚꽃 말고 http://www.melon.com/song/detail.htm?songId=4579468
이기주의보 http://www.melon.com/song/detail.htm?songId=30414360
랄랄라 송 http://www.melon.com/song/detail.htm?songId=30408194

시대별 차트

In [8]:
#http://www.melon.com/chart/age/list.htm?idx=1&chartType=AG&chartGenre=KPOP&chartDate=2000&moved=Y
In [11]:
def kpop_age_list(year):
    age_url="http://www.melon.com/chart/age/list.htm"

    params={
        "idx":1,
        "chartType":"AG",
        "chartGenre":"KPOP",
        "chartDate":1990,
        "moved":"Y",

    }
    html=requests.get(age_url,params=params).text
    soup= BeautifulSoup(html,'html.parser')

    for tag in soup.select('a[href*=playSong]'):
            title = tag.text
            js = tag['href']
            matched = re.search(r"'(\d+)'\);", js)
            if matched:
                song_id = matched.group(1)
                song_url  = "http://www.melon.com/song/detail.htm?songId=" + song_id
                print(title, song_url)
In [12]:
kpop_age_list(2000)
내사랑 내곁에 (응답하라 1988 삽입곡)  http://www.melon.com/song/detail.htm?songId=49933
너를 사랑해  http://www.melon.com/song/detail.htm?songId=57339
나의 노래  http://www.melon.com/song/detail.htm?songId=2981575
아주 오래된 연인들  http://www.melon.com/song/detail.htm?songId=50733
이별의 그늘  http://www.melon.com/song/detail.htm?songId=1695858
10년전의 일기를 꺼내어  http://www.melon.com/song/detail.htm?songId=49473
Poison  http://www.melon.com/song/detail.htm?songId=83721
내가 만일  http://www.melon.com/song/detail.htm?songId=1719746
도시인  http://www.melon.com/song/detail.htm?songId=49035
애모  http://www.melon.com/song/detail.htm?songId=1847750
너를 향한 마음  http://www.melon.com/song/detail.htm?songId=51976
이 밤이 지나도록  http://www.melon.com/song/detail.htm?songId=1839752
희망사항  http://www.melon.com/song/detail.htm?songId=52451
넋두리  http://www.melon.com/song/detail.htm?songId=53159
핑계  http://www.melon.com/song/detail.htm?songId=56870
신(新) 인류의 사랑  http://www.melon.com/song/detail.htm?songId=56245
사랑을 위하여  http://www.melon.com/song/detail.htm?songId=1877126
너에게 들려주고 싶은 이야기  http://www.melon.com/song/detail.htm?songId=61514
사랑일뿐야  http://www.melon.com/song/detail.htm?songId=53615
유리창엔 비  http://www.melon.com/song/detail.htm?songId=53088
달팽이  http://www.melon.com/song/detail.htm?songId=33189
시청 앞 지하철 역에서  http://www.melon.com/song/detail.htm?songId=3655353
잘못된 만남  http://www.melon.com/song/detail.htm?songId=62462
꿈에서 본 거리  http://www.melon.com/song/detail.htm?songId=1839784
사랑할수록  http://www.melon.com/song/detail.htm?songId=55991
달의 몰락  http://www.melon.com/song/detail.htm?songId=2183382
기억의 습작  http://www.melon.com/song/detail.htm?songId=61500
미소 속에 비친 그대  http://www.melon.com/song/detail.htm?songId=51481
나를 슬프게 하는 사람들  http://www.melon.com/song/detail.htm?songId=38662
또 다른 만남을 위해  http://www.melon.com/song/detail.htm?songId=463542
그 후로 오랫동안  http://www.melon.com/song/detail.htm?songId=58937
당신은 어디 있나요  http://www.melon.com/song/detail.htm?songId=65728
비오는 날 수채화  http://www.melon.com/song/detail.htm?songId=104134
언제나 그 자리에  http://www.melon.com/song/detail.htm?songId=91104
사랑을 할꺼야  http://www.melon.com/song/detail.htm?songId=1622501
오래전 그날  http://www.melon.com/song/detail.htm?songId=57583
마지막 콘서트  http://www.melon.com/song/detail.htm?songId=58160
김성호의 회상  http://www.melon.com/song/detail.htm?songId=50313
보고싶은 얼굴  http://www.melon.com/song/detail.htm?songId=1878281
넌 할 수 있어  http://www.melon.com/song/detail.htm?songId=38757
그대품에 잠들었으면  http://www.melon.com/song/detail.htm?songId=1010210
방황  http://www.melon.com/song/detail.htm?songId=91403
너를 사랑하고도  http://www.melon.com/song/detail.htm?songId=69663
너의 결혼식  http://www.melon.com/song/detail.htm?songId=91406
이 밤의 끝을 잡고  http://www.melon.com/song/detail.htm?songId=66261
마법의 성  http://www.melon.com/song/detail.htm?songId=61829
그녀를 만나는 곳 100M 전  http://www.melon.com/song/detail.htm?songId=51431
난 행복해  http://www.melon.com/song/detail.htm?songId=1839641
오늘 같은 밤이면  http://www.melon.com/song/detail.htm?songId=104668
보이지 않는 사랑  http://www.melon.com/song/detail.htm?songId=49541
그대안의 블루  http://www.melon.com/song/detail.htm?songId=111572
내게  http://www.melon.com/song/detail.htm?songId=532083
난 알아요  http://www.melon.com/song/detail.htm?songId=50012
널 사랑하니까  http://www.melon.com/song/detail.htm?songId=56288
먼 훗날  http://www.melon.com/song/detail.htm?songId=193014
슬픈 인연  http://www.melon.com/song/detail.htm?songId=62098
이별 아닌 이별  http://www.melon.com/song/detail.htm?songId=1878583
가려진 시간 사이로  http://www.melon.com/song/detail.htm?songId=1707087
살다보면  http://www.melon.com/song/detail.htm?songId=1698660
너에게 원한건  http://www.melon.com/song/detail.htm?songId=54227
내가 아는 한가지  http://www.melon.com/song/detail.htm?songId=45926
Dreams Come True  http://www.melon.com/song/detail.htm?songId=75068
그 아픔까지 사랑한거야  http://www.melon.com/song/detail.htm?songId=1017150
흩어진 나날들  http://www.melon.com/song/detail.htm?songId=51930
자아도취  http://www.melon.com/song/detail.htm?songId=1839185
나와 같다면  http://www.melon.com/song/detail.htm?songId=77363
삐에로는 우릴 보고 웃지 (Remake)  http://www.melon.com/song/detail.htm?songId=1104582
당신  http://www.melon.com/song/detail.htm?songId=1003590
늦지않았음을  http://www.melon.com/song/detail.htm?songId=49353
슬프도록 아름다운 ...  http://www.melon.com/song/detail.htm?songId=66052
이별의 끝은 어디인가요  http://www.melon.com/song/detail.htm?songId=65720
슬픈 표정 하지 말아요  http://www.melon.com/song/detail.htm?songId=53586
대답 없는 너  http://www.melon.com/song/detail.htm?songId=64056
내게로  http://www.melon.com/song/detail.htm?songId=61240
어떤이의 꿈  http://www.melon.com/song/detail.htm?songId=1104907
널 사랑하겠어  http://www.melon.com/song/detail.htm?songId=33206
하룻밤의 꿈  http://www.melon.com/song/detail.htm?songId=49838
난 널 사랑해  http://www.melon.com/song/detail.htm?songId=91114
꿈  http://www.melon.com/song/detail.htm?songId=1414786
칵테일 사랑  http://www.melon.com/song/detail.htm?songId=1931126
애송이의 사랑  http://www.melon.com/song/detail.htm?songId=70148
입영열차안에서  http://www.melon.com/song/detail.htm?songId=53616
난 멈추지 않는다  http://www.melon.com/song/detail.htm?songId=54414
걸어서 하늘까지  http://www.melon.com/song/detail.htm?songId=106047
립스틱 짙게 바르고  http://www.melon.com/song/detail.htm?songId=91116
외롭지만 혼자 걸을 수 있어  http://www.melon.com/song/detail.htm?songId=1104921
너를 처음만난 그때  http://www.melon.com/song/detail.htm?songId=49560
한사람을 위한 마음  http://www.melon.com/song/detail.htm?songId=1136819
영원  http://www.melon.com/song/detail.htm?songId=69198
그녀와의 이별  http://www.melon.com/song/detail.htm?songId=38547
그래도 이제는  http://www.melon.com/song/detail.htm?songId=91407
영원히 내게  http://www.melon.com/song/detail.htm?songId=64025
그대 슬픔까지 사랑해  http://www.melon.com/song/detail.htm?songId=91373
내가 너를 느끼듯이  http://www.melon.com/song/detail.htm?songId=49808
추억의 책장을 넘기면  http://www.melon.com/song/detail.htm?songId=64942
사계  http://www.melon.com/song/detail.htm?songId=53062
사랑의 서약  http://www.melon.com/song/detail.htm?songId=63343
어떤 이별  http://www.melon.com/song/detail.htm?songId=1517332
In [ ]: